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Overview 


Preparing high school students for both college and career is a goal that few can disagree with. But 
while much attention has focused on how to prepare students academically for life after high school, 
less has been directed at the nonacademic skills and knowledge that students need to succeed in 
college and the workplace. Schools are expected to teach these skills and knowledge, but they are 
rarely given the support, guidance, and tools needed to do so. 

Career academies — small schools within schools that are organized by a career theme — are 
particularly well positioned to provide these “21st-century skills.” Indeed, work-based learning 
experiences, such as internships, are a central, possibly an instrumental, component of the career 
academy model. Yet even career academies sftuggle to make college and career exploration and 
awareness-building curricula a central part of every student’s experience. 

With a grant from the Institute of Education Sciences in the U.S. Department of Education, MDRC 
and its project partner Bloom Associates piloted a program to help academies build college and 
career exploration programs. Called “Exploring Career and College Options (ECCO),” the program 
consists of a series of one-hour in-class lessons, visits to local work sites and college campuses, and 
a six-week internship with a concurrent weekly seminar that is offered in the summer before or 
during the senior year. If the program is fully implemented, by the time students graduate from an 
ECCO career academy, they will have received up to 44 lessons, participated in at least two visits to 
work sites and two to college campuses, and completed a six-week compensated internship. 

This report summarizes findings from a three-year study of 1 8 academies in three states — Califor- 
nia, Florida, and Georgia — that implemented ECCO from 2009 through 2012. 

Key Findings 

• ECCO significantly improved the capacity of career academies to offer college and career 
exploration curricula and activities. Academies with little or no existing capacity were able to 
launch all the components of the ECCO program within the first year. 

• Asa consequence of this increased capacity, students in ECCO academies participated in career 
and college exploration activities at substantially higher rates than students who were enrolled in 
the same academies before ECCO was implemented. 

• The ECCO academies were successful in placing into internships most of the students who were 
interested in and available for them. The internship component of the academy model is often 
viewed as the most challenging aspect of implementation. Surprisingly, when students did not 
participate in the internship program, they more likely opted out of it, rather than what has 
commonly been cited as the reason for low take-up rates — that not enough employers choose 
to host interns. The students’ reasons for opting out of an internship included mandatory sum- 
mer school and already having a summer job lined up. 
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Preface 


Operating within larger high schools, career academies are small, career-themed schools in 
which students take classes together as a cohort for three or four years and spend time in local 
work sites to experience firsthand the world of work. This approach is one of the most durable 
and popular high school reform models in the United States. Starting over 40 years ago in just a 
handful of schools, the career academy movement has grown to include over 7,000 academies, 
ranging from “green” academies in California to tourism academies in Florida to pre- 
engineering academies in the Northeast. Tens of thousands of students from a wide range of 
socioeconomic and racial/ethnic backgrounds, with different academic achievement levels and 
different educational and career aspirations, enroll in these schools. 

The career academy model is also one of the most studied of reforms. Two decades ago, 
MDRC launched the first random assignment evaluation of career academies to test their impact 
on academic, labor market, and other student outcomes. The results of that study, which showed 
that academies have strong and sustained efiects on earnings, have helped shape and fuel the 
growth of the academy movement ever since. One plausible interpretation of these findings is 
that it was the work-based learning component of the academy experience that drove these 
positive effects. In 2008, MDRC received a grant from the Institute of Education Sciences (lES) 
in the U.S. Department of Education to conduct a second impact study. 

The process of recmiting academies for that study, however, led to a surprising discov- 
ery: In nearly all the academies visited, the researchers learned that one of the key components 
of the academy model — internships and other work-based learning experiences — either was 
nonexistent or served only a handful of students. Although teachers and leaders believed in the 
value of work-based learning, they explained that they lacked the time, resources, and (some- 
times) the know-how needed to approach and recmit employer partners as well as to teach 
students about appropriate behavior at the work site. So MDRC switched gears and — with the 
support of lES — launched a different kind of project, whose goal was to create a program and 
curriculum that would build the capacity of academies to create strong work-based learning 
programs and to study what it takes for them to succeed. Because career academies are now as 
much about preparing students for postsecondary education as they are about preparing students 
to succeed in the workplace, a college exploration component was added to the program. 

Eighteen academies in six school districts volunteered to pilot the program, which was 
developed by Bloom Associates, Inc., in partnership with MDRC and which eventually was 
named “Exploring College and Career Options,” or “ECCO.” This report summarizes the 
findings and lessons learned from the academies’ experiences in creating from scratch or 
rebuilding a program that focused on work-based learning and college exploration activities. 
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The results show that, within just a year, academies can — with some help and support 
— build strong work-based learning and college exploration programs that reach most if not all 
of their students, rather than just a few. 

The number of academies continues to grow, thanks in large part to recent federal sup- 
port for this evidence-based model. The hope is that ECCO and the findings from this research, 
along with many other capacity-building efforts under way, will help academies fulfill their 
potential to make a difference for at-risk and low-income youth. 


Gordon L. Berlin 
President 
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Executive Summary 


Overview of the ECCO Program 

The phrase “preparing students for college and career” has become so ubiquitous that it has 
become almost a mantra in educators’ discourse in recent years. Whether mentioned in the 
Common Core State Standards, in the mission statements of high schools, or in political 
campaigns, improving the college and career readiness of young people is a concept that few 
can disagree with. Much attention has focused on how to prepare students academically for life 
after high school. But “readiness” also means having the knowledge and skills to make in- 
formed choices about careers and postsecondary education options and — once graduated — to 
successfully navigate both worlds. High schools are expected to teach these skills and 
knowledge but are rarely given the guidance or tools to do so. 

With a grant from the Institute of Education Sciences in the U.S. Department of Educa- 
tion, MDRC and its project partner Bloom Associates developed and piloted a program to help 
schools build or strengthen their college and career exploration programs. Called “Exploring 
Career and College Options (ECCO),” the program was designed specifically for career 
academies but can be adapted to fit many educational settings. Career academies are schools 
within schools that enroll up to several hundred students. They are organized by a career theme, 
such as health sciences or media arts. Besides regular high school courses, career academy 
students enroll in a sequence of career-technical courses centering on the theme area. Finally, 
students participate in internships and other experiences in workplaces — which is often called 
“work-based learning” — to reinforce the connections between what they learn in the classroom 
and their future careers. An earlier random assignment study of career academies conducted by 
MDRC demonstrated the effectiveness of the model. Over the years, as the number of career 
academies grew, the parallel pressure to ensure that all students meet high academic standards 
inadvertently crowded out time for career exploration activities — the very activities that 
nonexperimental evidence from the MDRC study suggests may have played an insfrumental 
role in causing the large increases in earnings that career academy participants experienced over 
the eight-year period following high school graduation. Career academies typically cite a lack of 
time, skills, and resources as the reason for not offering such activities to all of their students. 

ECCO is a capacity-building program to help career academies offer opportunities to 
students to learn about their workplace and postsecondary options through four core components: 
a series of one-hour in-class lessons, visits to local work sites, visits to college campuses, and a 
six-week internship offered to all students in the summer before or during their senior year. The 
curriculum includes guidance for educators on how to arrange and manage students’ out-of- 
school experiences as well as guides for partnering employers. As shown in Table ES.l, by the 
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The Career Academies Project 
Table ES.l 

ECCO Components, by Grade Level 


Semester 

ECCO Component 

lOtb- 

Graders 

lltb- 

Graders 

12tb- 

Graders Total 


Career exploration lessons 

5 

5 

10 lessons 

Fall 


Career exploration visits 

1 

1 

2 visits 

Fall and/or spring 

Career development lessons 

7 

7 

5 19 lessons 


College readiness lessons 

5 

6 

4 15 lessons 

Spring 


College visits 

1 

1 

2 visits 

Summer between 1 Itb and 12tb 

grades 

Internship and seminar 



1 1 internship 


NOTE; The number of lessons listed in the Total column is the maximum number of lessons available. Some 
lessons are considered optional. 


time students graduate from an ECCO career academy, they will have received up to 44 one- 
hour lessons, participated in at least two visits to work sites and two visits to college campuses, 
and completed a six-week compensated internship along with a concurrent weekly seminar. 

This report summarizes findings from a three-year study of the implementation of the 
ECCO program. ECCO was launched in 18 career academies in six school districts in three 
states: California, Florida, and Georgia.' The purposes of the study are to document the experi- 
ences of these schools in adopting the program and to assess the extent to which, when given 
support and resources, programs like ECCO can be frilly implemented. The study also collected 
descriptive data to assess the promise of the program to improve student participation in career 
and college exploration activities and to improve their awareness of postsecondary options. 


Key Findings from the Project 

• ECCO significantly improved the capacity of career academies to offer col- 
lege and career exploration curricula and activities to their students. Acade- 


'Nineteen academies volunteered to participate, but one academy withdrew before the program began. 
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mies with little or no existing capacity were able to launch all the compo- 
nents of the ECCO program within the first year. 

• As a consequence of this increased capacity, students in academies that im- 
plemented ECCO participated in career and college awareness and explora- 
tion activities at substantially higher rates than students who were enrolled in 
the same academies before ECCO was implemented. 

• Most of this increased participation in career and college exploration activi- 
ties occurred in the first two semesters of exposure to ECCO; an additional 
two semesters of exposure did not appear to substantially improve participa- 
tion beyond the boost achieved in the first year. 

• ECCO academies that were given generous support in the form of financial 
assistance and coaching were able to implement the program and sustain it 
with high fidelity for up to three years. Somewhat surprisingly, however, 
even those academies in the study that received less support were able to es- 
tablish or strengthen their programs. 

• The ECCO academies were successful in placing into internships most of the 
students who were interested in and available for them. The internship com- 
ponent of the academy model is often viewed as the most challenging aspect 
of implementation. Surprisingly, when students did not participate in the in- 
ternship program, they more likely opted out of it, rather than what has 
commonly been cited as the reason for low take-up rates — that not enough 
employers choose to host interns. The students’ reasons for opting out of an 
internship included mandatory summer school and already having a summer 
job lined up. 


Career Academies: A Proven High Schooi Reform Modei 

The academy model is one of the oldest and most stable of high school reform models in the 
United States. A number of factors help explain the lasting appeal of career academies. Especial- 
ly in recent years, academies focus on preparing students for both college and career, not one or 
the other, by offering not only a rigorous college preparatory curriculum but also career-technical 
education. Academy advocates reject the “false dichotomy” that students need to prepare for 
college or career, arguing that the skills needed to succeed in either arena are the same — or at 
least overlapping. Second, academies attract and are well suited to serve a broad cross-section of 
high school students, from those who are at high risk of dropping out to those who are on track to 
enroll in four-year universities. Finally, the career academy model has been the subject of much 
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research and evaluation, some of which has generated rigorous evidence of the model’s effec- 
tiveness and much of which has shaped the evolution of the model over the past few decades.^ 

MDRC’s pivotal study of career academies — started in the mid-1990s — was one of 
the most rigorous evaluations of any high school intervention conducted at that time.^ The study 
used random assignment to constmct a program group and a control group of about 1,800 
students and then tracked them through and beyond high school for several years after their 
scheduled graduation. The study found strong and sustained positive impacts from participation 
in career academies on students’ labor market experiences — notably, higher earnings. These 
impacts occurred without any detrimental effects on educational outcomes, such as graduation 
from high school or postsecondary enrollment.'' 


ECCO: A Career and College Exploration Program 

ECCO teaches students about the workplace and postsecondary options through a series of one- 
hour in-class lessons, visits to local work sites (called “career exploration visits’’) and to college 
campuses (“college visits’’), and a six-week internship offered to all students in the summer 
before or during their senior year. The curriculum includes guidance for teachers on how to 
teach the lessons, arrange and manage students’ out-of-school experiences, and work with 
employers who host the visits and accept interns. To ensure solid program implementation, 
ECCO includes a professional development and technical assistance component. 

As shown in Table ES.l, students who spend tenth, eleventh, and twelfth grades in an 
ECCO academy would experience — by the time of graduation — up to 44 lessons taught in 


^David Stem, Charles Dayton, and Marilyn Raby, Career Academies: A Proven Strategy to Prepare High 
School Students for College and Careers (Berkeley: University of California, Graduate School of Education, 
Career Academy Support Network, 2010), Web site: http://casn.berkeley.edu/resources.php?r=158&c=l. 

^James J. Kemple, Long-Term Impacts on Labor Market Outcomes, Educational Attainment, and Transi- 
tions to Adulthood (New York: MDRC, 2008). Web site: http://www.mdrc.org/sites/default/files/full_50.pdf 

''More recently, a study used state records data in California to describe the characteristics of students en- 
rolled in California Partnership Academies (CPAs) and compare them with those of students who were not 
enrolled in academies. The study also looked at differences in graduation rates and test scores. Although the 
study found that academy students graduated at a higher rate, the design did not permit controlling for students’ 
characteristics. But the study also compared the scores of academy and nonacademy students on the state’s 
standardized tests, controlling for both school and demographic characteristics. No significant differences were 
detected. See Charles Dayton, Candace Hamilton Hester, and David Stem, Profile of the California Partner- 
ship Academies, 2009-2010 (Berkeley: University of California, Graduate School of Education, Career 
Academy Support Network, October 2011), Web site: http://casn.berkeley.edu/downloads/CPA_Report_2009- 
lO.pdf; see also David Stem, Phil Saroyan, and Candace Hamilton Hester, Comparing Students in Each 
California Partnership Academy with Non-Academy Students at the Same High School, 2009-10 (Berkeley: 
University of California, Graduate School of Education, Career Academy Support Network, August 2012), 
Web site: http://casn.berkeley.edu/resource_files/ComparingAcad_%20Non-AcadStudents_2012.pdf 
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their regular career-technical or academic classes. They would have gone on two visits to local 
work sites and two visits to college campuses. In the summer between eleventh and twelfth 
grades, they would have had a six-week internship in a local work site and attended a weekly, 
half-day seminar covering a variety of topics that are conducive to success in internship, 
college, and career. 

A total of 1 8 career academies in five school districts volunteered to participate in this 
project. Four served as “pilot academies,” two of which operated the program for three years 
and two of which operated the program for two years. These four academies received substan- 
tial support in the form of free materials, training, regular coaching, and a grant to compensate 
the teacher who served as the ECCO coordinator in each academy. The remaining academies 
were recmited for a “scale-up phase” in which they were asked to operate the program support- 
ed by a less intensive “train-the-trainer” model of coaching, to assess the feasibility of scaling 
up the program in several academies at a time within a district. During the first year of ECCO 
implementation, for example, whereas the pilot academies received monthly calls and a visit 
each semester from the program developer, the scale-up academies received monthly calls and 
occasional visits from a district employee who was trained by the program developer, with 
periodic participation by the developer. 


Key Findings from the impiementation Study 

Overall, the pilot academies implemented the core components of ECCO as they were designed. 
All four academies taught the lessons in the proper sequence; arranged for all the required 
career exploration visits (in which at least 80 percent of their students participated); visited a 
range of postsecondary educational institutions, including both community colleges and 
universities; and placed in an internship about 70 percent, on average, of the students who were 
interested and available. 

This success was almost replicated by the scale-up academies using a train-the-trainer 
model and reduced support, although three of the fifteen academies did not complete the 
yearlong program. One academy withdrew before the project began, and two withdrew over the 
course of the year. The remaining scale-up academies were able to implement each component 
of the program with good fidelity. They conducted all the required off-campus visits and placed 
about the same percentage of their juniors in internships as the pilot academies did. But while 
the pilot sites taught all the required lessons, the scale-up sites taught, on average, about 80 
percent of the required lessons. 

Factors that appear to be associated with smooth program implementation across all the 
ECCO academies include: 
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• A strong district-level employee with the capacity and credibility to effec- 
tively support the academies 

• A high level of commitment to the goals of the program at the academy level 

• Alignment with the district’s existing initiatives and priorities 

• Flexibility in scheduling, within the academy’s calendar, coupled with re- 
lease time or stipends to compensate ECCO coordinators for the time they 
spent building and supporting partnerships with employers 


Key Findings from the Anaiysis of Participation Rates and 
Short-Term Outcomes 

The main outcome of interest is whether ECCO, a capacity-building intervention, resulted in 
increased student participation in career and college exploration activities. Surveys and a variety 
of other data sources were used to assess the evidence for this, as well as the extent to which 
ECCO could plausibly have improved other short-term outcomes, such as students’ awareness 
of college and career options and the connections between high school, postsecondary educa- 
tion, and their career aspirations. 

• Students with two semesters of exposure to the ECCO program participated 
in more career exploration activities than students in the same academies be- 
fore the program was implemented. For example, after two semesters of par- 
ticipation in ECCO, the percentage of students who visited a local work site 
as part of their school programming more than quadmpled. The percentage 
of students who performed a practice job interview more than doubled, and 
the percentage of students who reported that they had learned about what 
level of education is needed for various careers increased by 9 percentage 
points. (See Figure ES.l.) A second year of exposure to ECCO generally re- 
sulted in little additional growth in participation in these activities over and 
above the growth that happened in the first year.^ 


^These findings result from comparing participation rates at three points in time: the year prior to the acad- 
emy’s implementing ECCO, one year after implementation started, and two years after implementation. The 
first comparison (before and after one year of implementation) uses data on students in two academies: the 
Academy of Art and Technology (AOAT) and the Center for Hospitality, Tourism, and Marketing (HTM). 
The second comparison (between one and two years of implementation) uses data on students in two other 
academies: the Culinary Operations Academy (COA) and the Digital Safari Academy (DSA). 
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The Career Academies Project 
Figure ES.l 

Participation in Career Exploration Activities During the School Year: 

Comparison of Tenth- and Eleventh-Grade Cohorts Combined, with Zero and Two Semesters of ECCO. 

First- Year Academies (AO AT and HTM) 


□ Spring 2010 (N = 280) □ Spring 2010 (N = 274) 


Attended guest speaker presentation about jobs or workplaee skills 
Learned in elass about how to write a good resume 
Did a praetice job interview 

Learned in elass about how to aet appropriately in the workplaee 

Learned in elass about how to respond to workplaee ehallenges like 
diserimination/harassment 

Had opportunities to practiee or learn 2 1 st-eentury skills 
Diseussed with a teaeher or eounselor what career would be best for you 
Worked with a teacher or counselor to create a plan for your future 
Learned what level of education is needed for various careers 
Learned what level of education is needed for a career that interests you 
Visited a work site as part of a school activity 
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Figure ES.l (continued) 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on survey respondents enrolled in a given grade in spring 2010 and spring 201 1. The white bars are the regression-adjusted 
outcomes of students who responded to the spring 2010 student survey. The dark bars are the regression-adjusted outcomes of students who responded to the 
spring 2011 survey. 

Outcomes are adjusted for the following student characteristics: academy fixed effects, grade fixed effects, whether a student belonged to a racial or ethnic 
minority, gender, mother's educational attainment, whether a student was ever held back, whether the student received a D or below in English or math in the 
previous school year, and missing indicators for each of these characteristics. 

A two-tailed t-test is used for all statistical tests presented in this figure. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 
percent; * = 10 percent. 

Due to missing values on some survey items, the number of students included in the analysis varies by activity. The numbers of students reported in the 
figure are the number of students who responded to the survey. 

AO AT = Academy of Art and Technology; EITM = Center for Hospitalty, Tourism, and Marketing. 



• Similarly, students with two semesters in ECCO participated in substantially 
more activities to improve their understanding of their college options and to 
help them make informed choices. For example, after two semesters of par- 
ticipation in ECCO, the percentage of students who visited a four-year cam- 
pus doubled, and the percentage of students who reported that they had 
talked with a teacher about their college plans increased by 20 percentage 
points. Exposure to ECCO was also associated with a significantly higher 
percentage of students who attended a college fair, heard a presentation with 
college representatives, and talked with a teacher about how to pay for col- 
lege. (See Figure ES.2.) 

• Students with two semesters of participation in ECCO showed positive but 
modest growth on scales measuring awareness of career and college options. 


Conclusions 

The purpose of the project described in this report was to design and test an intervention that 
would systematically build the capacity of academies to offer career and college exploration 
activities for all their students. The primary question that drove the research is whether acade- 
mies — if given the resources and the support to do so — can launch and maintain a high- 
quality work-based learning program that includes all students. 

Nineteen academies in six school districts volunteered to implement ECCO — a pro- 
gram consisting of a sequenced set of lessons and off-campus experiences for students in tenth 
through twelfth grades who enrolled in career academies.*’ All but three academies — those that 
dropped out early in the project — were able to offer significantly more career and college 
exploration activities than they offered before ECCO. More students experienced more activi- 
ties. That being said, the work of building career and college exploration programs was not 
without significant challenges, even when academies had access to generous resources and 
support. Fitting new curricula into already-packed courses (whether it was a career-technical 
course or an English course), arranging visits to workplaces and college campuses, and finding 
suitable employers to serve as internship hosts took time and skill. Implementation was not 
perfect. But the accomplishments of the academies in implementing ECCO leave little doubt 
that the program succeeded in increasing their capacity to build career and college exploration 
programs that could reach all students rather than just a few, even for the group of academies 
that received less support. 


*’ECCO was designed for students in ninth through twelfth grades, but, for the purposes of this study, the 
academies were not required to include ninth-graders. 
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The Career Academies Project 
Figure ES,2 

Participation in College Exploration Activities During the School Year: 

Comparison of Tenth- and Eleventh-Grade Cohorts Combined, with Zero and Two Semesters of ECCO, 

First- Year Academies (AO AT and HTM) 


□ Spring 20 1 0 (N = 280) ■ Spring 20 1 0 (N = 274) 


Talked with a teacher or other adult at school about college 
Talked with a parent or other family member about college 

Attended a college fair 
Attended a presentation by college representatives 
Looked up information about colleges on the Internet 
Visited a 4-year college campus 
Visited a community college or career-technical school campus 
Talked with an adult about how to pay for college 
Took the P-SAT (or signed up to take it this summer) 
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Figure ES,2 (continued) 

SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on survey respondents enrolled in a given grade in spring 2010 and spring 2011. The white bars are the regression- 
adjusted outcomes of students who responded to the spring 2010 student survey. The dark bars are the regression-adjusted outcomes of students 
who responded to the spring 2011 survey. 

Outcomes are adjusted for the following student characteristics: academy fixed effects, grade fixed effects, whether a student belonged to a 
racial or ethnic minority, gender, mother's educational attainment, whether a student was ever held back, whether the student received a D or 
below in English or math in the previous school year, and missing indicators for each of these characteristics. 

A two-tailed t-test is used for all statistical tests presented in this figure. Statistical significance levels are indicated as follows: *** = 1 percent; 
** = 5 percent; * = 10 percent. 

Due to missing values on some survey items, the number of students included in the analysis varies by activity. The numbers of students 
reported in the figure are the number of students who responded to the survey. 

AO AT = Academy of Art and Technology; HTM = Center for Hospitalty, Tourism, and Marketing. 



While the study clearly establishes the feasibility of building the capacity of academies 
to offer career and college exploration activities and demonstrates that this increased capacity 
led to greater student participation in such activities, the evidence is less clear about whether 
increased participation in those new programs made a difference in the short-term outcomes that 
were predicted by the theory of change. An analysis of scores on scales measuring changes in 
career and college awareness, engagement, and 21st-century skills points to statistically signifi- 
cant and substantial differences for some indicators before and after students participated in 
ECCO. Without a more rigorous research design, however, it is difficult to attribute these 
differences to ECCO with confidence. 


Implications of the Findings for Policy and Practice 

The findings from this study of a capacity-building project to help academies offer their high 
school students a program of career and college exploration experiences suggest several 
implications for educators and researchers. First, while it is feasible to implement in academies 
a cohesive program of college and career exploration curricula and activities, this is unlikely to 
happen without support. Teachers need resources, time, and training. This is particularly tme for 
developing work-based learning programs that directly involve employers. Yet good results can 
be achieved in a relatively inexpensive “train-the-trainer” model of professional development 
whereby the school district provides most if not all of the direct support that academies need. 

Second, with resources and support, academies in this project were generally able to 
place about half of their juniors in internships. Although a handful of students were not placed 
because of a shortage of employer hosts, most of the juniors who did not participate opted out 
for reasons including obligations to attend summer school, the need to earn money, family 
summer plans, and the inability to pay for public transportation. This implies that alternatives to 
the traditional internship model — whereby one student is placed with one employer during the 
summer — should be considered. Many academies are already experimenting with such 
alternatives as “virtual internships,” group internships, employer mentoring programs, and 
school-based enterprises that arguably have the same desired learning outcomes of internships. 
Ideally, the traditional internship should be just one of several outcome-driven options that 
academies can offer to provide students with work-based learning experiences. 

Third, college and career exploration activities should be tightly integrated with the 
totality of the academy experience. These activities are best not relegated to the career- 
technical teacher alone. English, history, and math teachers should be involved with teaching 
an in-class career and college exploration curriculum, and they should find ways to integrate 
this material with their subject curriculum, accompany students on visits to the workplace and 
to college campuses, and create and monitor internships. When this occurs, students may be 
more likely to better understand the connections between what they are learning in their math 
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and English classes and their futures, leading to deeper engagement in school and better 
postsecondary outcomes. 

Fourth, preparing young people for success in the workplace and college is part of the 
mission of many programs serving young people, including regular high schools, extended 
learning programs, youth development programs, job training programs, community colleges, 
middle schools, after-school programs, and area vocational schools. ECCO — or programs like 
it — can be adapted and tested easily in a variety of such settings to learn what it takes to move 
beyond the rhetoric and actually bring effective services to low-income or at-risk young people. 

Fifth, many questions remain about the effects that career and college exploration and 
preparation programs in high school (offered both inside and outside career academies) have on 
long-term outcomes, such as enrollment in and completion of postsecondary education and 
participation in the labor market. While such programs hold promise to make a difference in the 
lives of at-risk, low-income students, without more rigorous evidence than is presented in this 
report, their tme potential remains unknown. 

For more information about the ECCO program and how to access it, see 
www.connectedstudios.org/ecco. 
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Chapter 1 

Introduction 

Overview of the ECCO Program 

The phrase “preparing students for college and career” has become so ubiquitous that it has 
become almost a mantra in educators’ discourse in recent years. Whether mentioned in the 
Common Core State Standards, in the mission statements of high schools, or in political 
campaigns, improving the college and career readiness of young people is a concept that few 
can disagree with. Much attention has focused on how to prepare students academically for life 
after high school. But “readiness” also means having the knowledge and skills to make in- 
formed choices about postsecondary education options and to learn the skills and behaviors to 
succeed in both college and careers. High schools are expected to teach these skills and 
knowledge but are rarely given the guidance or tools to do so. 

With a grant from the Institute of Education Sciences in the U.S. Department of Educa- 
tion, MDRC and its project partner Bloom Associates developed and piloted a program to help 
school districts and academies build or strengthen their college and career exploration programs. 
Called “Exploring Career and College Options (ECCO),”' the program was designed explicitly 
for career academies but is relevant and adaptable for many educational settings.^ Career 
academies are schools within schools that enroll up to several hundred students. They are 
organized by a career theme, such as health sciences or media arts. Besides regular high school 
courses, career academy students enroll in a sequence of career-technical courses centering on 
the theme area. Finally, students participate in internships and other experiences in workplaces 
— which is often called “work-based learning” — to reinforce the connections between what 
they learn in the classroom and their future careers. Career academies often stmggle to offer 
college and career exploration and awareness-building curricula and activities, and they typical- 
ly cite a lack of time, skills, and resources as the reason for the stmggle. 

ECCO’s goal was to build the capacity of academies to offer college and career explo- 
ration experiences to all their students from tenth through twelfth grades. The program com- 
bines an in-school curriculum with out-of-school experiences in postsecondary institutions and 
workplaces. To test the feasibility of implementing the program and to assess its success in 


'ECCO was first named “The Career Academies Project: Linking Education and Careers (CAP).” An 
ECCO teacher in the Oakland Unified School District — an ECCO district — suggested this name change. 

^One section of the curriculum, the Internship Seminar Lessons, was adapted from an existing curriculum 
created by the Institute for Research and Reform in Education (IRRE); MDRC and IRRE share the copyright. 
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increasing participation in the programs, ECCO was launched in 18 career academies in six 
school districts in three states: California, Florida, and Georgia.^ 


Key Findings from the Project 

• ECCO significantly improved the capacity of career academies to offer col- 
lege and career exploration curricula and activities to their students. Acade- 
mies with little or no existing capacity were able to launch all the compo- 
nents of the ECCO program within the first year. 

• As a consequence of this increased capacity, students in academies that im- 
plemented ECCO participated in career and college awareness and explora- 
tion activities at substantially higher rates than students who were enrolled in 
the same academies before ECCO was implemented. 

• Most of this increased participation in career and college exploration activi- 
ties occurred in the first two semesters of exposure to ECCO; an additional 
two semesters of exposure did not appear to substantially improve participa- 
tion beyond the boost achieved in the first year. 

• ECCO academies that were given generous support in the form of financial 
assistance and coaching were able to implement the program and sustain it 
with high fidelity for up to three years. Somewhat surprisingly, however, 
even those academies in the study that received less support were able to es- 
tablish or strengthen their programs. 

• The ECCO academies were successful in placing into internships most of the 
students who were interested in and available for them. The internship com- 
ponent of the academy model is often viewed as the most challenging aspect 
of implementation. Surprisingly, when students did not participate in the in- 
ternship program, they more likely opted out of it, rather than what has 
commonly been cited as the reason for low take-up rates — that not enough 
employers choose to host interns. The students’ reasons for opting out of an 
internship included mandatory summer school and already having a summer 
job lined up. 


^Nineteen academies volunteered to participate, but one academy withdrew before the program began. 
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Career Academies: A Durable and Effective Model for 
High School Reform 

ECCO is designed to work within a career academy environment/ Career academies were first 
developed some 40 years ago with the aim of restmcturing large high schools into small learning 
communities to create better pathways between high school and higher education and/or the 
workplace. Since then, the career academy movement has grown steadily. By 2008, at least 
7,000 academies were operating in hundreds of public high schools in the United States.^ 

Career academies operate as schools within schools, usually enrolling 30 to 60 students 
per grade. They typically start with the tenth grade, but some also enroll ninth-graders. Acade- 
mies are organized around broadly defined occupational sectors, such as health sciences, 
hospitality and tourism, law, and engineering, but students are not necessarily expected to find 
careers in these sectors. Instead, academies aim to expose students to an array of careers while 
preparing them to graduate from high school and seek postsecondary education. By showing 
students the relevance of what they are learning in schools to their futures and by giving them 
experiences that underscore those connections, academies seek to improve engagement in 
school. Academy students take classes together, remain with the same group of teachers over 
time, and follow a curriculum that includes both rigorous, college preparatory courses and a 
sequence of career-technical courses. Teachers of core subjects and teachers of career-technical 
courses are expected to collaborate in an effort to integrate their content and to emphasize the 
relevance of what they are teaching in these classes to their students’ future careers. Finally, 
students participate in work-based learning activities, such as summer internships, to reinforce 
those connections and to afford them the opportunity to learn skills for success both in the 
workplace and in postsecondary education. 

The academy model is one of the oldest and most stable of high school reform models 
in the United States. Unlike most reform models, the career academy model not only is durable 
but also has gained ground in the four decades since the first few opened their doors. A number 
of factors help explain the lasting appeal of career academies. Especially in recent years, 
academies focus on preparing students for both college and career, not one or the other, by 
offering not only a rigorous college preparatory curriculum but also career-technical education. 
Academy advocates reject the “false dichotomy” that students need to prepare for college or 
career, arguing that the skills needed to succeed in either arena are the same — or at least 
overlapping. Second, academies attract and are well suited to serve a broad cross-section of high 
school students, from those who are at high risk of dropping out to those who are on track to 


"^ECCO can be adapted to fit other educational settings, such as small thematic schools, community col- 
leges, extended learning, and youth development programs. 

^Stem, Dayton, and Raby (2010); Harvard University Graduate School of Education (2011). 
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enroll in four-year universities. Finally, the career academy model has been the subject of much 
research and evaluation, some of which has generated rigorous evidence of the model’s effec- 
tiveness and much of which has shaped the evolution of the model over the past few decades.*’ 

MDRC’s pivotal study of career academies — started in the mid-1990s — was one of 
the most rigorous evaluations of any high school intervention conducted at that time.^ The study 
used random assignment to constmct a program group and a control group of about 1,800 
students and then tracked them through and beyond high school for 12 years after their sched- 
uled graduation. The study found strong and sustained positive impacts from participation in 
career academies on students’ labor market experiences — notably, higher earnings. These 
impacts occurred without any detrimental effects on educational outcomes, such as graduation 
from high school or postsecondary enrollment.* 


The Challenges of Offering Work-Based Learning Experiences 

Work-based learning is a core component of the academy model, and although there is little 
rigorous evidence of its effectiveness as a stand-alone activity, it is widely viewed as a likely 
driver of the positive effects of the career academy model.'* Unfortunately, however, many of 
today’s academies struggle to create and sustain work-based experiences for their students. 
Their efforts to do so have been undermined by two federal policy developments. The School to 
Work Opportunities (STWO) Act, enacted in 1994, provided funding to school districts to 
support in-school and out-of-school career preparation curricula and activities, such as job 
shadows, internships, and school-based enterprises. But STWO “sunsetted” in the late 1990s, 
and funding for career-technical programs was also cut. Federal funding for summer job 
programs for low-income youth — often used by districts to help pay for work-based learning 
programs — was also drastically reduced in the 1990s. As a result, many school districts had to 
scale back their support of academies and similar programs. 

At the same time, the No Child Left Behind Act of 2001 drove schools to invest more 
time and resources in building students’ academic skills than their career-technical and work- 


^Stem, Dayton, and Raby (2010). 

’Kemple (2008). 

*More recently, a study used state records data in California to describe the characteristics of students en- 
rolled in California Partnership Academies (CPAs) and compare them with those of students who were not 
enrolled in academies. The study also looked at differences in graduation rates and test scores. Although the 
study found that academy students graduated at a higher rate, the design did not permit controlling for students’ 
characteristics. But the study also compared the scores of academy and nonacademy students on the state’s 
standardized tests, controlling for both school and demographics. No significant differences were detected. See 
Dayton, Hester, and Stem (2011); Stem, Saroyan, and Hester (2012). 

^Stem and Steams (2008); Page (2012). 
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place skills. With economic downturns in the past decade, resources for schools began to dry 
up, forcing them to cut electives and other nonacademic offerings. The result is that academies 
consistently report that they stmggle to incorporate work-based learning and college exploration 
activities into their programs, despite a strong desire to do so. For example, a national network 
of several hundred academies reports that as many as half of them failed to meet the standards 
set by the organization for a work-based learning program. 

It is this need that ECCO was designed to address: to increase the capacity of academies 
and similar programs to build (or, in some cases, rebuild) a program of in-school and out-of- 
school learning experiences to ensure that career and college awareness and exploration 
activities become a central part of every academy student’s experience. This report presents 
findings from a three-year pilot of ECCO, a capacity-building program. During the pilot, 1 8 
career academies in six school districts agreed to implement the program. By intention, the first 
4 of these 18 sites (referred to as the “pilot academies” in this report) received significantly 
more resources and support to implement ECCO than the other academies did. This was done 
so that the experiences of academies in implementing ECCO with varying degrees of support 
could be documented, in order to better understand what it takes to implement, sustain, and 
scale up the program in authentic settings. In addition, student-level data were collected during 
the two years in which the pilot phase occurred, to assess the potential of ECCO to lead to 
increased participation in college and career exploration activities and other outcomes that 
ECCO was designed to affect. The next section describes these outcomes in the context of the 
program’s theory of change. 


A Theory of Change 

ECCO teaches students about the workplace and postsecondary options through a series of one- 
hour in-class lessons, visits to local work sites and to college campuses, and a six-week intern- 
ship offered to all students in the summer before or during their senior year. The curriculum 
includes guidance for educators on how to arrange and manage students’ out-of-school experi- 
ences and guides for partnering employers. To ensure solid program implementation, ECCO 
includes a professional development and technical assistance component. 

ECCO has two overall goals: 

• To build the capacity of career academies and similar programs to make a 
continuum of work-based learning and college exploration experiences a 
central part of the experience of all academy students 


*°This estimate is based on annual self-assessments provided by the academies in 2010 and 2011. 
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• To increase participation in college and career awareness activities so as to 
improve students’ awareness and knowledge of their career and postsec- 
ondary options while helping them acquire the skills that they need to suc- 
ceed in both 

A premise of ECCO is that academies can rarely implement and sustain a continuum of 
high-quality work-based learning awareness, exploration and preparation learning experiences 
that are available to all students — not just a few — without resources and assistance. As 
illustrated by the theory of change depicted in Figure 1.1, such resources include release time or 
some form of compensation for an academy-level coordinator; district support; training and 
coaching; and materials, including curricula, tools, and guides. 

As shown in Figure 1.1, ECCO has four core components: ' ' 

1. A series of one-hour lessons that are taught in class by one or more acad- 
emy teachers and that cover a range of topics geared to teaching students 
about the workplace and their postsecondary options 

2. Annual half-day career exploration visits to local work sites related to the 
academy’s general career theme 

3. Annual full-day visits to a range of postsecondary institutions 

4. An internship program with a concurrent weekly seminar 

Academy activities begin in the fall semester of the first year and continue until the stu- 
dents graduate. The thinking behind ECCO is that, as a result of the academy’s strengthened 
capacity to ojfer these activities, more students will participate in them. Increased participation 
in these exploration activities is expected to heighten students’ awareness of the range of 
postsecondary options available and what is needed to prepare for and succeed in both college 
and career. In turn, seeing the relevance of education to their future careers will encourage 
students to become more engaged in school. As one ECCO student said in response to the 
question “What are you doing to prepare for the future?’’: “I am staying in school.’’ 


A Note on “21st-Century Skills” 

One of the goals of ECCO was to provide high school students with an awareness of and more 
opportunities to learn noncognitive, transferable skills — referred to in this report as “21st- 
century skills’’ and as “Skills for Success’’ or “Transferable Skills’’ in the ECCO materials. For 


"chapter 2 presents details about the content, sequencing, and structuring of the ECCO components. 
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The Career Aeademies Projeet 


Resources 


Release time or stipends for 

ECCO coordinators 

Support from school district: 

covers summer teacher, 
transportation costs, and 
employer contacts 

Training and site-level 
support: includes professional 
development and follow-up with 
regular phone coaching and 
monitoring 

Curricula and materials: 

includes lesson plans, employer 
host guides, student reflection 
logs, internship guide 


Figure 1.1 

ECCO: A Theory of Change 













ECCO, a set of 19 skills grouped into five categories was selected and adapted for high school 
students from several sources, including the Partnership for 21st Century Skills (P21) and the 
Secretary’s Commission on Achieving Necessary Skills (SCANS) of the U.S. Department of 
Labor. These skills are shown in Box 1.1. 


Box 1.1 

“Skills for Success” in the ECCO Program 

Creative and Critical Thinking 

Solve problems and make decisions 
Think creatively 
Learn through research 
Reflect and evaluate 

Communication 

Write clearly 
Speak distinctly 

Listen carefully and ask good questions 
Observe effectively 

Collaboration and Teamwork 

Cooperate with others 
Give and receive feedback 
Negotiate challenges and resolve conflicts 
Plan and prioritize to reach a goal 

Media and Technology 

Use new and emerging technology 

Use responsible behavior regarding technology 

Personal Qualities 

Manage time effectively 

Demonstrate appropriate behavior and attire 

Recognize and respond appropriately to workplace challenges 

Show initiative and work independently 

Be reliable and dependable 

NOTE: This set ofnoncognitive skills is adapted from skills defined in the Partnership for 21st Century 
Skills (P21) framework (Web site: http://www.p21.org/). 






One influential early book on this subject was published in 1996: Teaching the New 
Basic Skills, by Richard Mumane and Frank Levy. It named four “soft skills” needed in the 
workplace: the ability to solve semi-stmctured problems, whereby hypotheses must be formed 
and tested; the ability to work in groups with persons of various backgrounds; the ability to 
communicate effectively; and the ability to use technology.'^ A few years later, a group of 
federal agencies, major companies, and other organizations initiated the Partnership for 21st 
Century Skills. Included among the skills that P21 names as essential for students to learn are 
critical thinking, problem solving, creativity, communication, and collaborative skills.'^ A key 
goal of ECCO was to give students opportunities to improve such skills and to teach them that 
the skills are needed in order to be productive and successful, no matter which path they choose 
after high school. 


What This Study Can — and Cannot — Conclude About the 
Promise of ECCO to Affect Student Outcomes 

The primary purpose of this project was to develop and pilot a new program and then to 
document how it was implemented in career academies. Research was conducted to assess the 
extent to which the program was, in fact, implemented as designed — at least in a small number 
of schools. A second goal was to assess whether participation in ECCO holds promise for 
improving the student outcomes that the program is expected to affect. 

The most direct outcomes of interest, therefore, are whether ECCO, a capacity-building 
program, did increase the capacity of career academies to offer career and college exploration 
opportunities to their students and whether students participated in these activities at higher 
rates. The second question addressed by the project is whether there is any evidence that 
increased participation led to growth in the student-level short-term outcomes listed in the 
model (Figure 1.1). 

Note that the research was not designed to demonstrate whether there is — or is not — 
a causal link between the program and changes in student outcomes, for two reasons. First, 
testing such an inference would require a different design — such as a randomized controlled 
trial — to ensure that the students who received program services had the same characteristics 
before enrolling in ECCO as students who did not receive its services. Only through such a 
design can one isolate the effects of the program on outcomes. Second, by the time the project 
was over, most of the academies in the study had implemented the program for two years or 


'^Mumane and Levy (1996). 

'^For a summary of additional contributions to the call for including noncognitive skills in educational 
settings, see Darche and Stam (draft of June 2012). 
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less, and it may not have reached a high enough level of maturation or stability to warrant a 
rigorous impact evaluation. 

For these reasons, the goal in analyzing participation and short-term student outcomes is 
more modest. While an attempt is made to compare outcomes before and after the academies 
implemented ECCO, differences in those levels might have come about for a variety of reasons 
— ECCO being just one of them. 


A Project in Three Phases 

This project unfolded in three overlapping phases: 

• The first phase lasted one year (from 2009 to 2010) and focused on develop- 
ing the curriculum, other materials, and a professional development model. 

The materials were tested in two academies and then were modified several 
times, based on feedback from the students, teachers, and employers who 
had used them. 

• A two-year pilot phase (from 2010 to 2012) followed, in which four acade- 
mies (two of which also served as development sites in the first phase), im- 
plemented the nearly finalized curriculum and during which student-level data 
were collected to measure participation and learning outcomes. Two of these 
four academies started implementation a year after the other two academies. 

• A final, one-year scale-up phase (from 2011 to 2012) included three school 
districts, each of which implemented ECCO in several academies simultane- 
ously. These are referred to in this report as the “scale-up academies.” The 
report focuses only on the pilot and the scale-up phases, encompassing the 18 
academies that participated in the project. 

The Pilot Phase: Goals and Sites 

The primary goals of the pilot phase were to assess the feasibility of implementing the 
fully developed program in a small number of authentic settings and to document both the 
extent to which the academies increased their capacity to offer college and career exploration 
activities and the extent to which the rate of student participation in such activities increased. A 
before-and-after design was used to assess whether participation and learning outcomes 
changed in a manner that is consistent with the theory of change. 

'“'The curriculum developer and trainer for the project is Bloom Associates, Inc. 

*^As noted above, 19 academies volunteered to participate, but one withdrew before the program began. 
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As mentioned above, four academies served as pilot sites. All four volunteered to par- 
ticipate and were selected after being screened according to several criteria, including: 

• The main components of the career academy — with the exception of work- 
based learning — were reasonably well implemented and stable. 

• Academy leadership and teachers were eager to adopt the program and rec- 
ognized their need to build capacity in order to offer a work-based learning 
program to their students. 

• The host district was supportive of academies’ participating in the program, 
although direct district involvement was not required. 

• The academy admitted students without regard to prior academic perfor- 
mance, and the racial/ethnic makeup of academy students was not greatly 
different from that of the host high school. 

Table 1.1 presents selected characteristics of the four pilot academies and their districts. 
All four are located in large, comprehensive public high schools in urban school districts, as is 
typical of career academies. Two offered a media arts career theme; one offered a hospitality 
and tourism theme; and the fourth centered on culinary arts. The academies ranged in size from 
small to large; in spring 2010, the smallest (the Culinary Operations Academy) enrolled 68 
students, and the largest (the Academy of Art and Technology) enrolled 250. Three of the four 
academies were members of a state or national network of academies. 

As shown in Table 1.2, the pilot academies enrolled mostly underserved minority stu- 
dents, many of whom were academically at risk: About one-third had received a D or an F in 
English or math, and about one-fourth had been held back a grade. The raciaFethnic makeup of 
the students, however, was different across the academies, reflecting the demographics of the 
host high schools. For example, the participating students at the Center for Hospitality, Tourism, 
and Management are overwhelmingly African-American, and those at the Academy of Art and 
Technology are mostly Hispanic — as was the case at their respective high schools (Frederick 
Douglass High School in Los Angeles and Grover Cleveland High School in Atlanta). 

Academy-level characteristics are shown in Appendix Table A.5. The percentage of 
male students varied across the academies; The highest percentage of males was at the 
Culinary Operations Academy (60 percent), and the lowest percentage was at the Center for 
Hospitality, Tourism, and Marketing (36 percent). The Academy of Art and Technology had 
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The Career Academies Project 
Table 1.1 

Selected Characteristics of the Pilot Academies 



Academy of Art 
and Technology 

Culinary Operations Academy 

Digital Safari Academy 

Center for Hospitality, 
Tourism, and Marketing 

District 

Los Angeles Unified 
School District 

Hillsborough County 
Public Schools 

Mount Diablo Unified 
School District 

Atlanta Public Schools 

Location 

Los Angeles, CA 

Tampa, FL 

Concord, CA 

Atlanta, GA 

Career theme 

Media arts 

Culinary arts 

Media arts 

Hospitality 

Year established 

2004 

2002 

1996 

2003 

Grades served 

9-12 

9-12 

10-12 

9-12 

Total enrollment 

250 

68 

153 

167 

Minority students at host high school (%) 

84.7 

60.6 

81.8 

99.5 

Minority students at academy^ (%) 

89.2 

48.5 

88.2 

100.0 

Male students (%) 

50.2 

64.7 

49.0 

35.3 

Cohorting’’ 

Full 

Partial 

Full 

Partial 

Network membership 

CPA" 

NAF** 

None 

CPA" 

NAF"* 


SOURCES; Information was obtained from the 2009-2010 Common Core of Data and class rosters collected in spring 2010. 

NOTES; All four pilot academies are located within comprehensive high schools. Enrollment and demographic information are based on ninth- to 
twelfth- grade students enrolled in the academies in spring 2010. 

“This includes students who are black, Hispanic, Asian, or "Other" race/ethnicity. 

'’Cohorting refers to the co-enrollment of the same group of students in two or more classes. Academies strive to cohort all students in the same grade 
in at least one of their classes, but full, or "pure," cohorting is rare. Most academies manage to cohort students in at least the career-technical class and in 
one academic class, such as English. 

“California Partnership Academies. 

"^National Academy Foundation. 



The Career Academies Project 
Table 1.2 


Selected Characteristics of Enrolled Students and Survey Respondents, 

Pilot Academies 



Spring 2010 

Spring 2011 

Characteristic (%) 

Enrolled 

Students 

Survey 

Respondents 

Enrolled 

Students 

Survey 

Respondents 

Racial/ethnic minority'* 

87.3 

87.4 

91.6 

91.2 

Male 

48.3 

47.3 

45.4 

45.4 

Ever held back a grade 

NA 

22.6 

NA 

23.8 

Mother's educational attainment 

Less than high school 

NA 

43.8 

NA 

44.9 

High school diploma or GED certificate 

NA 

31.1 

NA 

32.4 

Postsecondary credential 

NA 

24.9 

NA 

22.2 

Received a D or below in English or math 

in the previous school year 

NA 

33.0 

NA 

34.5 

Grade level 

Grade 10 

39.9 

38.4 

37.2 

37.1 

Grade 1 1 

29.9 

30.1 

33.6 

34.4 

Grade 12 

30.2 

31.5 

29.1 

28.6 

Academy 

Academy of Art and Technology 

37.7 

38.9 

36.2 

37.8 

Center for Hospitality, Tourism, and Marketing 

28.9 

26.3 

31.5 

27.2 

Culinary Operations Academy 

10.4 

10.7 

8.9 

9.7 

Digital Safari Academy 

23.0 

24.1 

23.4 

25.3 

Total number of students'* 

695 

635 

666 

588 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: Values in the "Enrolled Students" columns are the characteristics of students in the spring academy 
rosters. Values in the "Survey Respondents" columns are the characteristics of students who responded to the 
student survey. Rounding may cause slight discrepancies in sums and differences. 

“This includes students who are black, Hispanic, Asian, or "Other" race/ethnicity. 

'’Due to missing values on some survey items and characteristics from the rosters, the number of students 
included in the analysis varies by characteristic. The total number of students reported here is for the full sample 
of students enrolled in the academies or who responded to the survey. 

NA = Data are not available. 
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the highest percentage of students whose mothers who did not complete high school (61 
percent) as well as the highest percentage of students who were academically at risk, as 
measured by low grades (45 percent). 

The Scale-Up Phase: Goals and Sites 

The goal of the scale-up phase was to assess the feasibility of scaling up ECCO in more 
than one academy at a time and with fewer resources than were used in the pilot phase. The 
scale-up phase began with site recruitment in spring 2011, with implementation beginning in 
fall 2011 and ending just before the following fall. Because the focus of the scale-up phase was 
on the ability of the districts and academies to implement the same program and not on the 
promise of the program for improving student-level outcomes, no student-level data were 
collected. Instead, academies were asked to complete a form each semester indicating their 
progress in implementation and their perceptions of how the program improved their capacity to 
offer career and college exploration opportunities to their students, compared with prior years. 

School districts were selected on the basis of their interest in and capacity for devoting 
time and resources to the project and on the availability of at least five academies with interest 
and capacity to adopt the program. The districts that were selected represented three different 
contextual settings that were expected to influence implementation. Miami-Dade County Public 
Schools is home to several dozen academies, the majority of which belong to the National 
Academy Foundation (NAF), a national network of about 400 academies. A NAF employee 
was assigned to partner with the district to support the academies in implementation instead. 
Hillsborough County Public Schools chose to participate because of the positive experience of 
one of the pilot sites that was located there (the Culinary Operations Academy) and because the 
program was aligned well with district strategic priorities. Finally, Oakland Unified School 
District was selected because this district was part of the statewide Finked Feaming initiative in 
California. The goals of Finked Feaming were also closely aligned with those of ECCO, 
making it a logical next step for the district to take. As a Finked Feaming district, Oakland 
Unified School District received support in the form of three paid “career technical specialists” 
to help the academies adopt the core components of the Finked Feaming model. 

Table 1.3 shows that the three districts each served a large city, one in California and 
two in Florida. Five academies participated in the ECCO program in two districts (Oakland 
Unified School District and Miami-Dade County Public Schools). Only four academies partici- 
pated in Hillsborough County Public Schools, since one academy dropped out before imple- 
mentation began. Miami-Dade was the largest district in the study, with 86 high schools and a 
total enrollment of nearly 1 00,000 students. Over 90 percent of the students who were served in 
Oakland and Miami-Dade districts belonged to a racial/ethnic minority group, and the majority 
of students in these two districts qualified for free or reduced-price lunch. 
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The Career Aeademies Project 
Table 1.3 

Characteristics of the Three Scale-Up Districts 


Characteristic 

Oakland Unified 
School District 

Hillsborough County 
Public Schools 

Miami-Dade County 
Public Schools 

Number of ECCO academies 

5 

4 

5 

Location 

Oakland, CA 

Tampa, FL 

Miami, FL 

Number of high schools in district 

31 

40 

86 

Total enrollment at high schools 

11,104 

53,410 

99,654 

Minority students in districf (%) 

91.1 

51.4 

90.4 

Free/reduced-price lunch (%) 

69.4 

44.0 

60.8 


SOURCE: Information was obtained from the 2009-2010 Common Core of Data. 

NOTE: ^This includes students who are black, Hispanic, Asian, or "Other" race/ethnicity. 


Data Sources 

A variety of data was collected, including student surveys, employer surveys, class roster data, 
interview data, and direct observations. (See Appendix A.) Program implementation was 
documented throughout the three-year project. More-detailed data were collected for the four 
pilot academies as well as student-level information to document participation in college and 
career exploration activities and to assess the learning outcomes of students with varying 
number of semesters in ECCO. It is especially important to note that two academies (the Digital 
Safari Academy and the Culinary Operations Academy) started implementation a full year 
before the other two academies (the Academy of Art and Technology and the Center for 
Hospitality, Tourism, and Marketing). 


Overview of the Report 

The remainder of the report is divided into five chapters: 

• Chapter 2 explains the ECCO model and its components in detail. Address- 
ing the question of what the program would look like if it were implemented 
with high fidelity, the chapter provides a context for Chapter 3. 
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• Chapter 3 describes in detail what the actual implementation of ECCO 
looked like in the four pilot academies. 

• Chapter 4 describes the experiences of the scale-up academies and their re- 
spective districts in implementing the program with significantly less support 
than the pilot sites received, relying instead on a train-the-trainer model to 
empower districts to support implementation of ECCO. 

• Chapter 5 presents findings based on student-level data collected from the pi- 
lot academies to assess the potential of ECCO to improve participation in ca- 
reer and college exploration activities and to promote growth in short-term 
outcomes — such as college and career awareness — predicted by the pro- 
gram model’s theory of change. 

• Chapter 6 concludes the report with some reflections on the implications of 
this study for educators and policies affecting schools’ efforts to prepare stu- 
dents for career and college. 
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Chapter 2 

The ECCO Model 


Chapter 2 describes in detail the model and key components of the Exploring College and 
Career Opportunities (ECCO) program. The purpose is to convey what ECCO should look like 
— when fully implemented — in order to provide a context for understanding how the program 
was actually implemented during the pilot and scale-up phases. 


Overview of the ECCO Program 

As depicted in the theory of change (Chapter 1, Figure 1.1), the ECCO model comprises a 
sequenced set of developmentally appropriate lessons that are taught during academy classes, a 
set of off-campus experiences in workplaces and college campuses, and a six-week internship. 
Designed specifically for career academies, the ECCO model can be implemented across all 
grades at the high school level, but the focus is on tenth, eleventh, and twelfth grades. 

The ECCO model consists of four components: (1) a series of one-hour lessons on top- 
ics that are designed to expand students’ awareness of and preparation for career and college; 
(2) sttuctured, half-day career exploration visits to local work sites related to the academy’s 
general career theme; (3) stmctured, full-day visits to a range of local postsecondary educational 
institutions; and (4) an internship program with a concurrent weekly seminar. Each component 
is described in detail below. 

One teacher in each academy — typically, the lead teacher (sometimes called the 
“academy director’’) or the career-technical teacher — serves as the ECCO coordinator. The 
coordinator is charged with overseeing the implementation of all four program components. 
Key responsibilities include identifying employer hosts for the career exploration visits and 
internships, arranging off-campus visits to workplaces and colleges, teaching the in-class 
lessons or overseeing other teachers who teach them, and working with the school district to 
coordinate logistics and resources. A premise of ECCO is that teachers need some form of 
compensation for the extra hours that it takes to build such a program; the compensation can be 
one period of release time or a stipend of a few thousand dollars. 

The coordinator is supported by training and ongoing support provided by the program 
developer or another qualified provider. A key premise of the ECCO project is that academies 
are rarely able to implement career and college exploration programs without support. To 
facilitate and support the operation of the program, ECCO also provides a framework, ready- 
to-deliver curricula, and guides for employers. Ideally, academies would have the resources to 
pay for training, followed by regular coaching and customized technical assistance. Because 


17 



most academies do not have these resources, a professional development component is built 
into ECCO that is meant to be both realistic and sufficient to ensure successful implementa- 
tion. It includes: 

• At least two half-day trainings provided by a qualified professional develop- 
ment provider or by trained school district staff 

• During the first year, monthly contact by telephone or site visits with a pro- 
vider or trained district staff 

• During the second year, at least one contact per semester with the provider or 
the district employee 

ECCO can be adopted by a single academy or by a school district in which several 
academies adopt the program at the same time. Some efficiency can be achieved when several 
academies in the same district implement the program simultaneously. For example, academies 
can be brought together for training. Such an approach may be more “sustainable” as well as 
more scalable than the more intensive, one-academy-at-a-time approach. Both models of 
implementation were piloted in this study and are covered in this report, in order to compare the 
implementation experiences of academies that had less access to the program developer and 
fewer resources than academies that received customized support and more financial resources. 
Academies that operated the program in a train-the-trainer framework were almost as successful 
as academies that received the more intensive, one-on-one support. 


A Closer Look at the ECCO Program Components 

Table 2.1 presents an overview of when the four components of the ECCO program occur for 
each grade level in an ECCO academy. As can be seen, students who spend tenth, eleventh, and 
twelfth grades in an ECCO academy would experience — by the time of graduation — up to 44 
lessons taught in their regular career-technical or academic classes. They would have gone on 
two visits to local work sites and two visits to college campuses. In the summer between 
eleventh and twelfth grades, they would have had a six-week internship in a local work site and 
would have attended a weekly, half-day seminar covering a variety of topics that are conducive 
to success in the internship, college, and career. 

The In-Class Lessons 

The in-class lessons of the ECCO program are taught by one or more academy teachers 
as part of their regular courses and teaching responsibilities. The lessons are designed to take up 
one class period of about 50 minutes. They can be taught exclusively in the career-technical 
course sequence or can be spread out among other courses, such as English, history, and math. 


18 



The Career Academies Project 
Table 2,1 

ECCO Components, by Grade Level 




10th- 

11th- 

12 th- 


Semester 

ECCO Component 

Graders 

Graders 

Graders 

Total 


Career exploration lessons 

5 

5 


1 0 lessons 

Fall 

Career exploration visits 

1 

1 


2 visits 

Fall and/or spring 

Career development lessons 

7 

7 

5 

1 9 lessons 


College readiness lessons 

5 

6 

4 

1 5 lessons 

Spring 

College visits 

1 

1 


2 visits 

Summer between 1 1th and 12 th 
grades 

Internship and seminar 



1 

1 internship 


NOTE: The number of lessons listed in the Total column is the maximum number of lessons available. Some 
lessons are considered optional. 


Since career-technical courses are often where students in career academies are exposed to 
career exploration activities as a cohort, it is in these courses that the ECCO curriculum is the 
most natural fit. Teaching students about their postsecondary options and the connections 
between high school, postsecondary education, and the workplace should be a shared responsi- 
bility, and so academic teachers are encouraged to teach some of these lessons. 

The lessons fall into three clusters: career exploration activities are geared to the work- 
place visits; college exploration activities are geared to the college visits; and career develop- 
ment activities prepare students for the internships and, more generally, for careers. Table 2.2 
lists the topics of each lesson in the three clusters. As mentioned above, if all the lessons are 
taught, students can participate in up to 40 lessons by the time they graduate from high school. 

The career exploration lessons are taught in the fall. Three lessons are taught before the 
workplace visits, and two are taught after them. These lessons have three learning objectives: 

• To prepare students for the career exploration visit, including researching in- 
formation about the company 

• To develop the skills to interact comfortably and confidently with working 
adults 

• To gain an awareness of the academic, technical, and personal skills required 
to succeed in the workforce 
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The Career Academies Project 
Table 2,2 

ECCO Topic Lessons, by Grade Level 


Career Exploration Activities 

Career Development Activities 

College Exploration Activities 

Grade 

(Fall Semester) 


(Fall and Spring Semester) 

(Spring Semester) 

10“ 

Lessons before the activity 

1. 

The endless job list 

Lessons before the visit 


1. Career explorations: 

2. 

Taking a stand on beliefs 

1. What is college: myths and 


What? Where? Why? 


about work (optional) 

facts 


2. Skills for Success 

3. 

The big picture (optional) 

2. Researching a college and 


3. Making a professional first 

4. 

Guest speaker (optional) 

planning a visit 


impression 

5. 

Planning a resume 

Lessons after the visit 


Lessons after the activity 

6. 

Writing a resume 

3 . Reflections on the visit 


4. What does the activity 

7. 

Portfolios: what and why 

4. Planning for college 


mean to me? 



5. Introduction to College Board 


5. Expressing appreciation 



resources (optional) 


(optional) 
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Same as above 

1. 

The endless job list 

Lessons before the visit 



2. 

Planning a resume 

1. What is college: myths and 



3. 

Writing a resume 

facts 



4. 

Creating and using business 

2. Researching a college and 




cards 

planning a visit 



5. 

Why do an internship, 

Lessons after the visit 




anyway? 

3. Reflections on the visit 



6. 

The winning interview 

4. Choosing the right college 



7. 

Portfolios: what and why 

5. Guest speaker: college 





entrance exams 





6. Preparing for your own college 





visit 

12 

Same as above 

1. 

Choosing a path that’s right 

Lessons before the individnal 




for you 

visit 



2. 

Career academy Skills for 

1. Planning ahead for college 




Success revisited 

2. Applying to college 



3. 

Writing a winning cover 

3. Financial aid: guest speaker 




letter (optional) 

Lesson after the individnal visit 



4. 

Individual job shadows 

4. Finish strong: making a 



5. 

Your exit portfolio 

smooth transition 


NOTE: “The tenth-grade lessons can be adapted for the ninth grade. 

The college exploration lessons are taught in the spring. Two lessons are taught before 
the campus visit, and three are taught after it. These lessons have four learning objectives: 
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• To examine career and life goals and to articulate clearly how college helps 
achieve those goals 

• To understand the opportunities and challenges of college as a next option 

• To be able to discuss the barriers to attending college and to brainstorm strat- 
egies and solutions for overcoming them, based on firsthand testimonies 

• To learn key terms for college planning 

The career development lessons are spread out across the three grades and can be taught 
in the fall or spring semester. The lessons follow a prescribed curriculum and have many hands- 
on experiences to engage the students. There are six learning objectives: 

• To expand awareness of the many jobs and careers available in the career 
academy’s theme area 

• To explore the relationships between interpersonal skills, good work habits, 
and success in the world of work 

• To learn and practice literacy skills (speaking, listening, reading, writing) 

• To address skills for success needed in school and at work, such as creative 
thinking, critical thinking, communication, collaboration and teamwork, un- 
derstanding new and emerging technology, reflection, self-assessment, and 
time management 

• To develop and refine a resume 

• To assemble a “Career and College Portfolio”' 

The Career Exploration Visit 

The career exploration visit is a half-day, off-campus experience whereby all the stu- 
dents in tenth and eleventh grades visit a work site that performs work related to the broad 
theme of the career academy. For example, students in a Health Sciences Academy could visit a 
pharmaceutical lab or a local hospital. Students in a Hospitality and Tourism Academy could 
visit a hotel or a tourist attraction. Students in a Green Academy might visit a recycling facility. 
The purpose of these highly stmctured events is to offer students an opportunity to learn 


*The portfolio, assembled by students, should contain examples of the student’s best work, including a 
resume, a cover letter, writing samples, and pictures or descriptions of products or projects completed at school 
or in the internships. 
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firsthand about the workplace and to interact directly with people who work in their chosen 
career theme area. The ECCO model requires that participating academies take their students on 
at least one visit for tenth-graders and another for eleventh-graders each year. 

The career exploration visit has four learning objectives: 

• To expand students’ awareness of careers and work environments 

• To make connections between what is learned at school and what is expected 
in the workplace 

• To observe how employees apply job-specific skills, how transferable skills 
are used in a variety of jobs, and how technology is used in the workplace 

• To practice the skills needed to interact comfortably and confidently with 
working adults 

In contrast to more typical field trips or traditional job shadows, students and employers 
are well prepared for career exploration visits, and there are explicit activities to be included in 
each visit, such as a hands-on activity, lunch with employees, and small-group discussions. 
Employer hosts are provided a written guide ahead of time to help them plan for the academy’s 
visit. Conversely, as part of the lessons described above, students conduct prior research on the 
host organization, prepare questions to ask employees, and practice behaviors appropriate for 
interacting with adults in the workplace. 

The College Visits 

The ECCO program includes at least two group visits to colleges each year, one visit 
for tenth-graders and another for eleventh-graders. While many high schools organize campus 
tours for some of their students, the expectation for an ECCO academy is that all students visit 
colleges. Other defining differences in the ECCO model are that the college visits are custom- 
ized to align with the career theme of the academy and that students have some direct experi- 
ences with college classes and students. Finally, ECCO students visit a variety of college 
campuses — including community colleges, proprietary schools, and four-year universities — 
to expand their awareness of all their options, including those that may initially be viewed as 
“out of reach.” 

The college visits have six learning objectives: 

• To reinforce the idea that college is a viable option for all students 

• To provide firsthand exposure to college environments — classes, living ar- 
rangements, student services, admissions, clubs, campus facilities, and so on 
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• To introduce students to a range of postsecondary institutions — two-year 
and four-year, residential and commuter, larger and smaller, urban and cam- 
pus, and so on 

• To help dispel myths or misconceptions that students may have about college 

• To prepare students for additional college visits that they will make 

• To make face-to-face connections with individuals and campus personnel for 
future follow-up by students 

The Internship Program 

The internship and seminar component is considered to be the capstone of ECCO; the 
lessons and the career and college exploration visits are meant to prepare students to succeed in 
their internship. 

The goals of the internship program are to give students the following opportunities: 

• To connect what is learned in school with how it is applied in the workplace 

• To become more excited about career opportunities and related postsecond- 
ary options 

• To increase students’ confidence by working and networking with local pro- 
fessionals 

• To become familiar with and practice 21st-century skills (Chapter 1), such as 
teamwork, time management, creative thinking, and problem-solving 

• To become familiar with workplace such issues as diversity, leadership, and 
stress management 

All students who are available and interested must be offered an internship. As subse- 
quent chapters show, many students opt out of summer internship programs due to competing 
demands on their time, such as mandatory credit recovery programs or jobs that they have 
already lined up. On average, however, about half of all juniors opted into the internship 
program. The internship takes place in the summer between the junior and senior years, al- 
though academies are encouraged to offer internships during the senior year if students are not 
available in the preceding summer. Internships last approximately six weeks, and students are 
expected to work a minimum of 20 hours per week. Academies are strongly encouraged to offer 
paid internships but are not required to do so. All students who complete the program must be 
awarded at least one academic credit. 
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The ECCO internship model is designed to make the experience as rich a learning expe- 
rience as possible for the student, while at the same time creating a product or service that has 
concrete value to the employer. In fact, employers and interns alike are instructed that the 
internship is not a job but, rather, a learning experience for the student. To reinforce classroom 
learning with the learning in the workplace, the internship should be relevant to the student’s 
skills and interests and should fit the career theme of the academy. ECCO coordinators attempt 
to match students with specific placements, and students who are identified as potential candi- 
dates are required to interview for the internship. Coordinators are expected to visit each 
internship site at least twice a summer, to ensure that the learning objectives are being met and 
that students as well as their supervisors and coworkers are having a positive experience. 

To help determine whether an employer or organization would be able to provide good 
learning opportunities for students, ECCO coordinators are given a tool consisting of a series of 
questions that can be used both to stmcture the initial conversation when meeting with prospec- 
tive hosts and to develop a suitable project for the student to work on. In addition, employers or 
organizations that are selected to be internship hosts are given a guide that outlines the roles and 
responsibilities of both ECCO interns and internship hosts and that provides tips and ideas to 
help hosts create the best possible learning environment for each intern. 

Working with their supervisor, with coworkers, and sometimes with other interns from 
the same academy, students take on a long-term project or a series of tasks that, when complet- 
ed, yield a tangible product, such as brochure, a video, or a document. Students bring these 
products of their internship to a summer-ending celebration and public presentation to which 
their supervisors, parents, and teachers are invited. 

Students who are in the internship program also attend a concurrent seminar that is con- 
ducted one day per week for all six weeks. The seminar is a key mechanism whereby students 
can maximize their work-based learning by engaging in reflection and research. Taught by one 
of the academy teachers at the school or at another location that is convenient for the students, 
the seminar covers six of the following eight topics: first week in the internship, time manage- 
ment, managing stress, diversity in the world of work, customer service, job satisfaction, 
leadership roles, and moving forward. Each seminar follows the same format, including a 
check-in with the students about how the week is going, a time when students can report out on 
the research assignments they received the week before, engagement with a set of activities 
covering the topic of the week, and concluding with a research assignment. Each intern main- 
tains a “Research and Reflection Eog” that links the topic of the weekly seminar to the work 
that they are doing in their internship. 
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Chapter 3 

The Implementation Experiences 
of the Four Pilot Academies 


Introduction 

Chapter 3 describes how four pilot academies implemented the Exploring Career and College 
Options (ECCO) program model described in Chapters 1 and 2. It addresses the question of 
whether the pilot academies — given the support, training, and resources that they received — 
were able to implement ECCO and the extent to which the program enhanced their capacity to 
offer career and college exploration opportunities to their students. The chapter also describes 
the challenges and accomplishments of operating the program in the ever-changing and unpre- 
dictable context of high schools. 

The chapter draws on data collected during the three-year pilot phase of ECCO — 
from 2009 through 2012 — including annual student surveys, class rosters, site visits, inter- 
views, records kept by the site coordinators, and surveys of student interns and their employer 
supervisors. (Appendix A describes the quantitative and qualitative data sources used in 
Chapters 3, 4, and 5.) 


Key Findings 

• Overall, the pilot academies implemented the core components of ECCO 
with a high degree of fidelity. 

• ECCO resulted in an increase in both the quantity and the quality of college 
and career exploration activities for students in the pilot academies. 

• Teachers and students in the pilot academies were enthusiastic about the 
ECCO program — particularly, the off-campus work-based learning experi- 
ences and the visits to college campuses. 

• The internship program was the most challenging ECCO component to im- 
plement, but the pilot academies still managed to greatly improve the reach 
and quality of their internships. Surprisingly, most of the academies found 
more employer hosts than there were available students who could accept an 
internship. 
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Overview of the Pilot Academies 

As explained in Chapter 1, four academies were selected to participate in the pilot phase of 
ECCO: the Culinary Operations Academy in Tampa, Florida; the Digital Safari Academy in 
Concord, California; the Academy of Art and Technology in Los Angeles; and the Center for 
Hospitality, Tourism, and Marketing in Atlanta, Georgia. These four academies varied in size, 
career theme, and the demographic characteristics of their students. (See Table 1.1 in Chapter 
1.) What the pilot academies had in common when they were recmited for this project was a 
need for and strong interest in strengthening programming for career and college readiness and 
exploration activities — particularly, internships. All four were eager to get help to do what they 
knew already they should be doing as academies: provide work-based learning experiences to 
most if not all students. Brief profiles of each academy follow. 

The Culinary Operations Academy (COA) 

The Culinary Operations Academy (COA) at Chamberlain High School in Tampa, 
Florida, was founded in 2002. Serving approximately 68 students in grades 9 through 12 (as of 
spring 2012), the academy offered a rigorous curriculum focused on the food service and 
hospitality industries. The curriculum addressed both state career and technical education 
standards and the standards set by ProStart, a program of the National Restaurant Association. 
Participation in ProStart helps this academy’s programs meet industry standards and offers 
students the opportunity to receive certifications recognized by the industry. At the time that the 
academy joined the project, it had several employer partnerships, the strongest of which was 
with Outback Steakhouse. Outback had even built a scaled-down version of one of its restau- 
rants in the high school, complete with a kitchen, which provided a real-world learning envi- 
ronment for the students. The academy had not offered an internship program in the past, and 
both the school district and the academy viewed ECCO as an opportunity to do so. 

The Digital Safari Academy (DSA) 

The Digital Safari Academy (DSA) at Mt. Diablo High School in Concord, California, 
offered an interdisciplinary approach to learning through a focus on multimedia arts and 
computer science. As of spring 2010, the Digital Safari Academy served approximately 153 
students in grades 10 through 12. Founded in 1996, it is one of the oldest academies in the 
California Partnership Academies state initiative.' The academy curriculum emphasized 
project-based learning and offered students various opportunities to tackle real-world problems. 


'state legislation launched the California Partnership Academies (CPAs) initiative in 1984. By 2010, ap- 
proximately 300 CPAs were operating in California public high schools. See the initiative’s profile on the 
California Department of Education Web site: http://www.cde.ca.gov/ci/gs/hs/cpareport09.asp. 
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including an annual Innovation Fair for which student teams formed virtual companies, created 
new technology products, and developed business plans. Academy leaders were assisted in 
implementing ECCO by a longtime volunteer from the community who was a small business 
consultant and former employer. The academy welcomed the opportunity to develop an 
internship program, stating that it had not had one for many years and that it considered this to 
be a top priority for its programming. 

The Academy of Art and Technology (AOAT) 

The Academy of Art and Technology (AOAT) at Grover Cleveland High School in Los 
Angeles, California, was founded in 2004 and, as of 2010, served approximately 250 students in 
grades 9 through 12. Focused on the visual arts, this academy offered a thematic, project-based 
curriculum for students that incorporated technology at all levels. It emphasized rigor and 
college preparation; every student enrolled in an advanced placement class during the senior 
year. The academy’s work-based learning program before ECCO was minimal. It had organized 
a few field trips to local employers but had never operated an internship program. 

The Center for Hospitality, Tourism, and Marketing (HTM) 

The Center for Hospitality, Tourism, and Marketing (HTM) at Frederick Douglass High 
School in Atlanta, Georgia, was founded in 2003 and, as of spring 2010, served 167 students in 
grades 9 through 12. As a member of the National Academy Foundation, this academy offered a 
curriculum including marketing, finance, and key principles of customer service, and it strove to 
provide a rigorous and relevant curriculum to assist students in meeting the demands of the 
industry and postsecondary institutions. The academy was eager to launch ECCO and, like the 
other pilot academies, saw this as an opportunity to build its capacity to offer internships. 
Before ECCO, few, if any, if its students had participated in an internship. 


The Implementation of ECCO’s Core Components 

Although ECCO offers lessons and activities for students in grades 10 through 12, because the 
plan for each pilot academy was to phase in activities over two years, each of these academies 
focused on the lessons and activities for tenth- and eleventh-graders in the first year, and they 
added lessons and activities for twelfth-graders in the second and third years of the pilot. 

The In-Class Lessons 

As described in Chapter 2, ECCO contains a series of one-hour lessons that are taught 
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in one or more of the academy courses for each grade throughout the year.^ Some lessons in the 
pilot were clustered around the career exploration visits or around the college visits; some 
lessons (the “career development lessons”) were freestanding and could be scheduled whenever 
convenient for the school. Teachers who were assigned to deliver the lessons were expected to 
fit them into existing curricula. All four pilot academies taught the lessons and sequenced them 
as intended, although some teachers found it difficult to find time for this, particularly in the 
program’s first year. 

The pilot academies generally taught the career development lessons and conducted the 
career exploration visits and their accompanying lessons in the fall. In the spring, they typically 
taught the college visits and their accompanying lessons. The in-class lessons were taught 
during the school day — usually in the career-technical class — but, by the second year of 
implementation, three of the four academies began using subject-matter teachers in the academy 
to teach some of the lessons, easing the burden on the career-technical teacher. In the second or 
third year of implementation, all the ECCO coordinators found it easier to fit the ECCO lessons 
into their regular curriculum. In one academy, however, the students were not “cohorted” — 
meaning that there was no single class in which all the students in that grade were enrolled — 
the teacher sfruggled to deliver the lessons to the students for whom they were designed. Of the 
54 eleventh-graders in that academy, only 27 were gathered in one class at the same time, and 
so only these cohorted students were taught the ECCO lessons. As a result, some eleventh- 
graders were not eligible to participate in the off-campus activities. 

All four of the ECCO coordinators reported that their students appeared to be engaged 
in the lessons. One teacher said: “After teaching about the professional handshake in ‘The 
Winning Interview,’ I now greet my students at the door to welcome them and shake their 
hands, so they get the practice.” Another teacher wrote: “Just having the curriculum is useful. It 
is extremely useful. It makes me go on up and beyond what I would normally do.” Several of 
the coordinators stated that the ECCO lessons provided a more coordinated approach to teach- 
ing about college and career awareness than what they had done previously. 

The Career Exploration Visit 

Having a job involving multimedia is way different than just having class. Now 1 
know that it’s a lot harder because things are for real life. The projects they do 
for other people have to be perfect and precise. 

— An ECCO student 


^ECCO lessons and activities that are designed for tenth-graders can also be offered to ninth-graders. The 
Culinary Operations Academy included ninth-graders in ECCO beginning in its second year of implementa- 
tion; the other three pilot academies either did not enroll ninth-graders or chose not to include them in ECCO. 
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A career exploration visit is a structured half-day visit to a local employer that exposes 
students to careers and professionals working in the broad occupational area relating to the 
academy’s theme. Before going on the visit, students learn about how to interact with the 
employees, prepare questions, and conduct research on the organization. The two lessons that 
are scheduled after the visit help students reflect on their experiences and learning. In the ECCO 
model, all tenth- and eleventh-graders (and, in some cases, ninth-graders) participate in at least 
one career exploration visit every year. 

This component of ECCO was implemented with very high fidelity among the pilot 
academies and was often identified by the coordinators and students as the most engaging of all 
the ECCO activities. As was planned, all four of the academies conducted annual career 
exploration visits for tenth- and eleventh-graders each year. These visits were generally well 
attended; typically, 80 percent to 100 percent of students in the targeted grades participated. 

Although all four of the pilot academies had conducted a few field trips to work sites 
before ECCO, none had attempted to implement visits as consistently or in as stmctured a 
manner as was required by the program. Employer hosts were asked to provide a very different 
type of learning experience for the students — not just a formal tour, but an opportunity to see 
firsthand employees at work and to talk with them about the education and training that they 
needed and how they felt about their work. ECCO provided a “Guide for Employer Hosts,” 
which coordinators reported was a valuable tool; they used it to approach prospective employer 
hosts, to introduce their academy, and to talk about the desired learning outcomes for the career 
exploration visit. The employer hosts praised this guide for so clearly outlining the expectations 
for the visit. 

The stmcture of career exploration visits was prescribed, and most of the visits were 
conducted as planned. All included time for students to tour facilities or to meet employees 
individually or in small groups. The ECCO model includes a hands-on experience or — in 
workplaces where this was impossible — a small-group discussion in which students could 
interact directly with employees while observing them working. In a Thank You letter written 
after the visit, one student reflected on what she had learned from interacting with employees: “I 
now have a better understanding of how everyone in the business works together and depends 
on one another to get the job done and please the client. ... I fruly appreciated the honest 
opinions you gave us and the encouragement to go to college and make a future for ourselves.” 

In interviews, all four of the ECCO coordinators said that the career exploration visits 
were successful in achieving their goals (listed in Chapter 2). Additionally, three of the four said 
that the visits were of “much higher quality” than any work-based learning experiences that had 
been offered in the years before ECCO. The preparation and reflection lessons seemed to be a 
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key factor in that success; several employer hosts noted how well prepared the students were 
when they arrived for the visits. 

The marketing director of a high-profile public relations agency that hosted a career ex- 
ploration visit said: 

We thoroughly enjoyed having the students visit our facility. They were amaz- 
ingly well prepared, well behaved, and seemed to be interested in the jobs and 
processes we perform. Tour guides and hosts of various departments were very 
impressed with the quantity and quality of questions from the students. 

Box 3.1 describes a typical career exploration visit as experienced by students at one of 
the pilot academies. 

The College Visits 

The college visit has . . . encouraged me to not just try to survive through high 
school by passing my classes but to thrive through high school by trying much 
more challenging courses and to learn valuable skills while 1 am still in high 
school. 

— An ECCO student 

The ECCO model (Chapter 1, Figure 1.1) includes visits to colleges or universities that 
are sfructured much like the career exploration visits, including opportunities to interact with 
college students, to observe classes, to visualize themselves in college, and to understand the 
breadth of their postsecondary options. In addition to meeting with college admissions staff, 
some college visits included such activities as touring a department related to the career acade- 
my’s theme, lunching with academy alumni on campus, or observing a class in the same theme 
area as their academy. As with the career exploration visits, students are well prepared by three 
one-hour lessons before a college visit; after the visit, they receive two additional lessons to help 
them reflect on what they learned. (Chapter 2, Table 2.2, lists the topics of these lessons.) 

To ensure that students’ experiences went well beyond a typical group tour of college 
and thus to comply with the ECCO model, coordinators reported working hard to arrange the 
college visits. This component was implemented with very high fidelity across the pilot acade- 
mies. All four of them taught the lessons in the correct sequence and conducted college visits 
with tenth- and eleventh-graders; some conducted additional visits for ninth- or twelfth-graders. 
The college visits were a significant addition to the pilot academies’ programming; few had 
conducted such visits before implementing ECCO. All four coordinators reported in interviews 
that this ECCO component increased the number of students who had an opportunity to visit one 
or more colleges. Through planning the visits and teaching the lessons, several of the academies 
increased their involvement and interaction with the college counseling services provided at the 
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Box 3.1 


A Typical Career Exploration Visit: 

The Culinary Operations Academy’s Visit to Outback Steakhouse 

Outback Steakhouse’ s corporate offices had supported the Culinary Operations Academy for 
several years by donating funds for a model restaurant set up at the academy. So this was a 
logical business to approach about hosting a career exploration visit, and it eagerly took on 
the role. On the morning of the visit, students, teachers, and chaperones drove in private cars 
to the nearby restaurant. Upon arriving, the group of approximately 25 participants was 
seated in the restaurant, where the manager welcomed them. He provided each student with a 
copy of the employee manual that is given to all new hires. The manager then addressed the 
key values of the Outback Corporation. The students were excited to note how well aligned 
they were with the Skills for Success that had been taught in the ECCO lessons. 

After the introductory session with the manager, the class was split into smaller groups for a 
detailed tour that included every department: the hostess stand, bar, food preparation areas, 
grill, and so on. Following the tour, each student was assigned to work side-by-side with a 
designated employee in one area of the restaurant. One small group accompanied the manag- 
er on his rounds; they tasted every item for a last-minute quality-control test. Some students 
cooked steaks on the line at the beginning of the meal-time msh. The students found this 
hands-on experience almost overwhelming, as the pace and volume far exceeded anything 
that they had experienced before. 

Students were enthusiastic about their experiences. Outback Steakhouse employees were 
impressed as well. Following the visit, the restaurant took on several Culinary Operations 
Academy students as summer interns. 


host high school. As a result, academy coordinators and teachers developed an increased sense of 
responsibility and expanded their capacity to help students understand college options. 

For many students, this was the first time that they had visited a college campus. As one 
student put it: “I got to see what a real college is like, and I also got the chance to see how 
people act and what’s expected of them.” In the students’ written reflections, they expressed the 
importance of the experience in their lives, summed up as follows: “If I really want to become 
someone in life, I have to be determined and not give up. ... I will meet my goals and prove to 
everyone [that] I am capable of anything.” 

Box 3.2 describes a typical college visit as experienced by students at one of the pilot 
academies. 
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Box 3.2 


A Typical College Visit: 

The Academy of Art and Technology’s Visit to Cal State, Northridge 

On a warni April morning, Academy of Art and Technology’s junior class boarded buses to 
visit California State University’s Northridge campus. Although the campus is less than five 
miles from the academy, only a handful of the students had ever been there. They were 
greeted by a former student of the academy who was now attending the university and who 
took them on an in-depth tour of the art department. They stopped to talk with a printmaking 
professor and a member of the photography faculty. They then dropped in at studios, offices, 
and classrooms. Throughout the tour, their host told them about the classes that she had taken 
and the projects that she had completed with the support of her instructors. She also told them 
that their education at the Academy of Art and Technology was preparing them well for Cal 
State, Northridge. 

Following the tour of the art department, the students split up into small groups for a full- 
campus tour led by three student guides from the admissions office. They visited the library, 
the gymnasium, and the cafeteria. Then they gathered in the lecture hall for a frank talk with 
a current student; she gave them tips on admission and financial aid and talked about her 
experiences adjusting to college. She shared some mistakes that she had made and outlined 
what she was doing to succeed. Following the tours, the students headed off in small groups 
to eat lunch and explore the campus on their own. 

Returning to the academy, the students voiced a newfound understanding and appreciation 
for what it meant to find the right college — and to plan ahead. Their teachers were im- 
pressed with the students’ new awareness and motivation. “Most of our students think of the 
local community college as their only option. We wanted this to be a reality check: The 
university has resources and can offer a level of opportunity that the community college 
cannot. We want them to know that they can aim higher — and now they do.” 


The Internship Program and Seminars 

1 would recommend the internship program because it really helps you improve 
your skills. It could also be the path to your future college and future career. 

— An ECCO intern 

The ECCO model includes the expectation that an internship will be offered to all stu- 
dents during the summer following their junior year or during their senior year. This was an 
intentionally ambitious goal, set high to learn what it takes to realize this key component of the 
academy model. As Chapter 1 notes, creating internships for high school students has been 
historically difficult. And when schools do operate such programs, sometimes only the best- 
behaved or most academically advanced students are invited to participate. Creating internships 
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that not only are available to any student who wants one but also are focused on learning 
outcomes adds to the challenge. 

As described in Chapter 2, the ECCO internship emphasizes student learning that en- 
hances both technical and 21st-century skills while helping students see the connections 
between what they are learning in school and what is needed to succeed in the workplace and in 
their futures. These outcomes are achieved through specific components of the internship 
experience: student projects, a student research and reflection log, and a weekly seminar taught 
by the ECCO coordinator or other academy instmctor during the internship. Overall, the four 
academies in the pilot phase greatly increased their capacity to offer internships to more 
students than they had previously served, and many of the placements met or exceeded the 
standards for an ECCO internship, as described in detail below. 

Establishing the Internship Program: Finding Hosts and Placing Interns 

The ECCO coordinators spent several months preparing for the summer internship pro- 
gram. Following a daylong training led by the program developer early in the year, the coordi- 
nators began by conducting outreach to parents and employers to inform them about the 
internship opportunity. School districts helped out when they could by providing lists of 
potential employer hosts in the academy’s career area. Academies consulted their advisory 
boards (if they had them) for leads, sent postcards and e-mails, and made cold calls to organiza- 
tions that they found online.^ Seeking hosts for internships, coordinators approached a wide 
variety of types of companies and organizations, including both locally and nationally based 
businesses, nonprofit organizations, and government agencies. Several coordinators found 
success in unlikely places by “thinking outside the box.” For example, the coordinator at the 
Digital Safari Academy found that there were not enough multimedia companies nearby to host 
all of its interns but that many local nonprofits were desperate for workers who had multimedia 
skills; so the coordinator placed many of the interns with these nonprofit employers, explaining: 
“We were able to help the students with any technical questions, and working in a nonprofit 
gave them the kind of work experience and mentoring we wanted them to receive.” 

As shown in Table 3.1, in three of the four pilot academies, ECCO coordinators con- 
tacted between 28 and 38 employers, and one academy reached out to as many as 180 employ- 
ers. When an employer expressed interest in participating, the coordinator arranged an in-person 


^Many career academies set up advisory boards of local employers and sometimes postsecondary institu- 
tions, and this is, in fact, required for academies in the National Academy Foundation and the California 
Partnership Academies. The boards were used sometimes to help with identifying employers, but the acade- 
mies typically found that the leads that the boards provided were not sufficient to generate enough suitable 
placements. 
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The Career Academies Project 
Table 3.1 


Internship Program Development, from Initial Contacts with Employers to 
Internship Placement, Pilot Academies 



Academy of 
Art and 
Technology 

Center for 
Hospitality, 
Tourism, and 
Marketing 

Culinary 

Operations Digital Safari 
Academy Academy Average 

Number of initial contacts with employers 

38 

31 

28 

180 

69 

Number of contacted employers who hosted 
career exploration visits, internships, 
or both 

21 

11 

23 

41 

24 

Number of internships developed 

26 

9 

21 

66 

31 

Contacts that developed into internships (%) 

68 

29 

75 

37 

44 


SOURCE: MDRC calculations are from ECCO coordinators' annual reports from 2010 and 2011. 
NOTE: A few employers hosted more than one student intern. 


meeting to answer questions and explain what was expected of the employer. These conversa- 
tions were often guided by an ECCO protocol to gauge the suitability of the site for a learning- 
focused internship and to help develop the student intern’s project. The coordinators reported 
that the protocol was helpful but, in some cases, was too time-consuming and detailed. Several 
coordinators adapted it so that the interview would take less time but would still help them 
convey the principles of an ECCO internship to the employer. 

Once a pool of employers agreed to host interns, coordinators then set about matching 
individual students to employers. This process involved aligning students’ needs, interests, 
personalities, and abilities with the specific needs and capabilities of each internship host. 
Students were then referred for interviews and were placed only if both the employer and the 
prospective intern agreed that the placement was a good match. 

Figure 3.1 shows the intake, participation, and completion rates for student interns 
across the four pilot academies. According to the spring class rosters from 2010 and 2011, there 
were 308 eleventh-graders enrolled in the four academies in the spring semesters. Of those, 
about half were available and interested in participating in an internship. (Reasons for opting out 
of internships are discussed below.) Of the 128 students who opted into the internship program. 
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The Career Academies Project 


Figure 3.1 

Student Intake and Participation in the Internship Program: 
Pilot Academies, 2010 and 2011 Combined 



SOURCES: The number in the first box, “1 Ith-graders,” was calculated from the class roster data and eombines 2010 and 
20 1 1 data. The numbers in the subsequent boxes were calculated from records kept by ECCO eoordinators and collected in 
both 2010 and 2011. 

NOTE: “This includes all reasons that a student did not participate, including the need to attend credit recovery programs, 
earn money in summer jobs, and transportation costs. 
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106 (83 percent) were interviewed and were placed; of those, 84 percent (89 students) complet- 
ed the program/ One academy experienced considerably more difficulty than the other three 
with recmiting and placing students in the internships, bringing the average placement rate 
down. As shown in Appendix Table B.l, this academy placed only 41 percent of the 22 students 
who were interested and available; the other three placed much higher percentages of their 
available eleventh-graders, ranging from 80 percent to 92 percent. 

These participation rates pertain to the internship programs that were operated in the 
summers of 2010 and 2011. All four pilot academies ran the program a third summer, but fewer 
data were collected to document the experience. Even so, the data are sufficient to suggest that 
the pattern of participation is similar to the pattern for the first two years. The placement rate is 
somewhat lower. Based on interviews conducted shortly before the project ended late that 
summer, approximately 60 of the 163 eleventh-graders in the four academies were available and 
interested in participating, and, of those, 57 were placed. Close to 88 percent of those who were 
placed completed their internships. 

Challenges in placing students in internships are often attributed to a failure to find 
enough employers to host interns rather than to students (or their parents) opting out. This study 
points to a more complex set of reasons why more students did not participate. Across all four 
pilot academies, only half the juniors wanted to or could participate in the program. In fact, in 
some cases, there were more employers willing to host a student than there were interested and 
available students. Interviews and survey data point to several reasons why some eleventh- 
graders chose not to participate. Table 3.2 shows results of a survey that was administered a few 
weeks before the internship programs were scheduled to begin; students were asked about their 
interest in participating, what they expected to get out of the experience, and their reasons for 
not planning to participate if that was their choice. Nearly all (97 percent) of the eleventh- 
graders surveyed said they were interested in having an internship, and 74 percent of them said 
that they would still be interested even if they were not paid.^ About one-third (31 percent) said 
that they would not participate because their family needed them at home, and 26 percent said 
that they would not participate because they already had a job lined up for the summer. When 
asked why students opted out of the program, some coordinators said that many students had to 
enroll in credit recovery or remedial programs in summer school. 


"^The rate of students opting into the internship program was slightly higher at the Culinary Operations 
Academy and the Digital Safari Academy (approximately 50 percent) than at the Academy of Art and 
Technology and the Center for Hospitality, Tourism, and Marketing (approximately 33 percent). 

Wll the internship programs offered academic credit. Academies varied on their policy for offering mone- 
tary compensation. In one academy, all but a handful of placements were paid internships. In another, only 
unpaid internships were offered. The remaining two offered a mix of paid and unpaid internships. In place of 
wages, one academy was able to provide some one-time stipends to students, ranging from $400 to $500. 
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The Career Academies Project 
Table 3.2 


Students Plans for Participating in Summer Internships, 
Pilot Academies 


Indicator (%) 

2010 
1 Ith-Graders 

2011 
1 Ith-Graders 

All 

Students** 

Student is interested in having an internship in future 

97.4 

97.3 

97.3 

Student would want to do an internship even without pay 

71.6 

75.4 

74.3 

Parents would probably want student to do an internship 

82.4 

88.2 

86.6 

Student might not be able to do internship because family needs 
student at home 

28.0 

31.8 

30.7 

Student cannot do internship this summer because a job is 
aready lined up 

20.0 

29.0 

26.3 

Total number of students'’ 

79 

202 

281 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: “Responses are averaged across summer 2010 and summer 2011. Two academies (Academy of Art 
and Technology and Center for Hospitality, Tourism, and Marketing) started offering summer internships in 
summer 201 1, so these academies have a smaller weight in the averages. 

’’Due to missing values on some survey items, the number of students included in the analysis varies by 
measure. The total number of students reported in the table are the number of eleventh-graders who responded 
to the surveys. 


Given the main reasons why students opted out of the internship program, it is not sur- 
prising that the characteristics of those who opted in and were placed and who completed it 
were different from the characteristics of those who did not apply — even though coordinators 
were instmcted not to intentionally screen students for the program. Table 3.3 confirms that 
their characteristics differed. For example, students who opted into and completed the program 
were more likely to be female and were less likely to have been held back a grade, and their 
mothers tended to have higher educational attainment. The successes of the internship program 
that are described in the next section may be due at least in part to the types of students who 
participated. 

Experiences in Implementing the Internship Program and Seminars 

The ECCO internship model is designed to ensure that the internship is well stmctured 
and that the intern receives appropriate tasks and guidance, works on a long-term project, and 
has opportunities to reflect on and learn from the work experience — both in terms of technical 
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The Career Academies Project 
Table 3.3 


Characteristics of Students Participating in Summer Internships 
(Among Survey Respondents), Pilot Academies 


Characteristic (%) 

1 Ith-Graders 

Applied to 
Internship 

Placed in 
Internship 

Completed 

Internship 

Racial/ethnic minority'* 

89.2 

90.4 

88.6 

88.5 

Male 

49.3 

39.2 

41.0 

35.6 

Mother's educational attainment 

Less than high school 

44.1 

38.3 

33.3 

30.7 

High school diploma or GED certificate 

31.7 

35.5 

35.6 

36.0 

Postsecondary credential 

24.2 

26.2 

31.1 

33.3 

Ever held back a grade 

22.2 

14.5 

11.5 

9.3 

Received a D or below in English or 

math in the previous school year 

28.1 

26.3 

24.1 

24.7 

Academy 

Academy of Art and Technology 

27.0 

23.8 

23.6 

26.1 

Center for Hospitality, Tourism, and Marketing 

21.0 

15.9 

8.5 

6.8 

Culinary Operations Academy 

14.2 

16.7 

18.9 

20.5 

Digital Safari Academy 

37.7 

43.7 

49.1 

46.6 

Total number of students'* 

281 

126 

106 

88 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 


NOTES: Values in the " llth-Graders" column are the characteristics of eleventh-grade students in spring 2010 
or 201 1. Values in the "Applied to Internship Program" column are the characteristics of students who applied 
to participate in a summer internship in summer 2010 or 2011. Values in the "Placed in Internship" column are 
the characteristics of students who were placed in internships in summer 2010 or 201 1. Values in the 
"Completed Internship" column are the characteristics of students who completed their internship in summer 
20 1 0 or 20 1 1 , so these academies have a smaller weight in the averages. 

“This includes students who are black, Hispanic, Asian, or "Other" race/ethnicity. 

'’Due to missing values on some survey items and characteristics from the rosters, the number of students 
included in the analysis varies by characteristic. The total number of students reported here are the numbers of 
students who responded to the surveys. 

skills and the less tangible but equally important 21st-century skills. (Chapter 1, Box 1.1, lists 
the “Skills for Success” that ECCO emphasized.) In their responses to a survey administered at 
the end of the internship, both interns and their supervisors indicated that, for the most part, 
these aspects of the ECCO internship model were implemented well. As one employer put it: 

This is my second year hosting . . . interns. This year’s experience was better 
than last year’s. The program helped us understand the skill set that [the acade- 
my] is developing. 1 have a sense that the experiences and opportunities that we 
provided were beneficial to the interns in ter ms of their understanding of local 
government and the importance of a client’s communication needs. 
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A key feature of the ECCO internship model is the student project, which is supposed 
to involve a service or product that brings value to the host employer while providing a mean- 
ingful work experience to the student. (Box 3.3 presents some examples of the types of projects 
that interns worked on.) All ECCO interns were assigned projects or a series of related tasks — 
some developed by the coordinator and the employer, some designed by the employer, and 
several involving student input. Some students had to balance daily tasks with the ongoing 
project, while others worked almost exclusively on one large project or several smaller projects 
for the duration of the internships. Many students cited the project experience in particular as a 
highlight of their internship, and most completed the internship with tangible evidence of their 
work — such as sample text, menus, videos, or brochures — which was showcased at a public 
presentation at the end of the summer. An employer noted how the interns gained both technical 
skills and some of the 21st-century skills emphasized in the ECCO project-centered internship: 
“The interns provided extremely valuable foundational research and graphics for the launch of 
this new project. They are talented and motivated. It was pleasure to see them grow their 
confidence and skills.” 

Another key element of the ECCO internship model is structured interactions between 
interns and supervisors in a manner that supports student learning and increases the intern’s 
chances of success during the internship. For example, the program required employer supervi- 
sors to meet the interns early in the placement to explain what was expected of them. Employers 
were also instructed to check in regularly with interns, just as a typical supervisor does normally 
to answer questions, monitor progress, and help solve problems. Table 3.4 presents survey 
responses from both interns and supervisors, who agreed that these meetings occurred and were 
a valuable resource to help maximize student learning. In fact, several supervisors indicated that 
they wished that the programs for their college-age interns had included the same approach to 
learning and performance. One employer said: “We had full support with a professional 
working environment, excellent equipment, immediate troubleshooting, and dialogue about 
ideas if we asked for it.” 

A few employers said that more information about the skills that students brought to the 
placement would have helped them better prepare more appropriate tasks and projects for the 
interns to work on. As one said: “It would be helpful for the proprietor to have a fuller under- 
standing of the school curriculum and the classes that the students have completed. For instance, 
what fundamental skills would each student possess at the first, second, and third year in the 
program so that we could gauge and prepare our training to complement it.” 

The ECCO internship model includes the expectation that the coordinators would visit 
the work sites regularly throughout the summer. The monitoring visits allowed coordinators to 
offer encouragement and support to interns and their supervisors, to assist in resolving problems 
before they became serious, or to facilitate additional learning. Coordinators visited each work 
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Box 3.3 


A Sample of Internships and Student Projects 

Georgia Lighthouse Foundation 

The intern designed and created an exhibit for a display space in the foundation’s new build- 
ing. This “Wall of Fame” was developed to honor the volunteer doctors who served the 
foundation; it included photos and biographical sketches of the doctors as well as testimoni- 
als from clients. 

Boys and Girls Clubs 

The intern interviewed past and present participants of boys and girls clubs to learn about 
their programs and activities and then used the information to create public service an- 
nouncements, local newspaper articles, and photo essays. 

Sabine Harte Architects 

The intern assisted in all phases of the design of a monument, from meeting with the donor 
who co mmi ssioned the monument to drafting it in software programs. He presented various 
drafts to a review board of his colleagues and received feedback. At the end of the summer, 
the design was accepted by the client, who praised the intern for his “passion and sensitivity.” 

Doorstep Farmers 

The intern completed several multimedia design projects for this new, youth-based business 
promoting organic food. The intern designed a business card for the president of the organi- 
zation, a pocket-size brochure with advice about eating organic food, a Web page, and a 
coupon that brought in new customers. 

Pittsburg Art and Community Foundation 

The intern created both a Web-based and a print brochure for a self-guided public art walk in 
the town. The task included taking and cropping photos, laying out the map, and writing 
explanatory text and instructions. The intern also created a video to promote the foundation’s 
storytelling program; the video was used in a grant application. 

GrillSmith 

The intern analyzed the time spent when using weights rather than volume measurements in 
recipes. She tested sample recipes, documented her findings, and presented her recommenda- 
tion for using weight measurements. The intern then assisted in converting the recipes, and 
the new method was put into operation at all six GrillS mi th locations. 
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The Career Academies Project 
Table 3.4 


Implementation of Internship Program Practices as Reported 
by Interns and Supervisors, Pilot Academies 


Item 

Percentage 

Intern^ reports 


Interns reported that supervisor: 


Interviewed intern before internship began 

88.8 

Oriented intern at start of internship 

89.9 

Met regularly with intern to discuss work 

77.5 

Assisted intern in completing assignments for seminar 

75.3 

Attended or planned to attend intern's closing ceremony 

69.7 

Coordinator visited work site: 


Not at all 

13.3 

1 -2 times 

47.0 

3-4 times 

20.5 

5 times or more 

19.3 

Number of coordinator visits was: 


Too few 

25.0 

Too many 

3.6 

Just about right 

71.4 

Total number of interns* 

90 

Supervisor^ reports 


Employers reported that they: 


Met with coordinator to plan intern tasks and project 

64.2 

Interviewed intern before internship began 

88.7 

Oriented intern at start of internship 

98.1 

Met regularly with intern to discuss work 

92.5 

Assisted intern in completing assignments for seminar 

77.4 

Attended or planned to attend intern's closing ceremony 

66.0 

Coordinator visited work site: 


Not at all 

3.8 

1 -2 times 

34.6 

3-4 times 

46.2 

5 times or more 

15.4 

The number of coordinator visits was: 


Too few 

2.0 

Too many 

2.0 

Just about right 

96.0 

Total number of supervisors* 

53 


SOURCES: MDRC calculations are from the intern and employer surveys. 


NOTES: Responses are averaged across summer 2010 and summer 2011. The Academy of Art and 
Technology and the Center for Hospitality, Tourism, and Marketing started offering summer internships in 
summer 2011, so these academies have a smaller weight in the averages. 

“Due to missing values on some survey items, the number of students and supervisors included in the 
analysis varies by measure. The total number of interns and supervisors reported in the table are the number of 
students or supervisors who responded to the surveys. Approximately 50 percent of all supervisors responded 
to the survey. 
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site an average of two or three times, although visits were more sporadic at one academy. The 
coordinators generally found the visits to be very helpful, and most interns and supervisors 
reported that they found the number of visits “about right.” 

The Internship Seminar 

A critical feature of the ECCO internship is the concurrent, three-hour, weekly seminar, 
which was usually taught at the school. This gave student interns an opportunity to meet as a 
group with their teacher to learn strategies and skills to succeed in the workplace and to discuss 
the accomplishments and challenges that they were experiencing in their internships. According 
to interviews conducted with the program developers and the coordinators, the seminar lessons 
were delivered as planned and were well received by the students. As shown in Table 3.5, 
almost all of the students who responded to a question on the survey described the seminars as 
either “somewhat useful” or “very useful.” The coordinators who taught the seminars reported 
that students seemed to benefit from them in several ways. For example, students were encour- 
aged to bring up the challenges that they were facing in the workplace before a situation became 
overwhelming, so that the teacher and other students could suggest solutions. As one coordina- 
tor explained: “We used the time in two ways. We let them talk through problems or concerns 
they had with their supervisors and coworkers; we also answered any technical questions they 
had, because in some cases their supervisors didn’t have the time or the technical expertise.” 

Interns, coordinators, and supervisors were typically highly satisfied with the internship 
program. For example, interns and supervisors almost unanimously reported in surveys that the 
work performed by the intern was beneficial both to the intern and to the organization. Across 
the four pilot academies, 92.9 percent of students liked their internship; 89.3 percent reported 
that they learned a lot; and (as is discussed in Chapter 5) 87 percent felt that their 21st-century 
skills improved as a result of the experience (Table 3.5). Nearly all the supervisors (92.5 
percent) would consider hosting another intern. In their reflections on the internship experience, 
students expressed greater self-confidence, increased motivation for school, and renewed hope 
about their future. As one student said: “It’s an eye-opening experience and shows you that all 
your courses in school are relevant.” Another said: “It was good to be able to get in real-world 
workplaces and learn how it feels to basically be an adult. We got to be in an internship that 
could open our opportunities for the future with connections, skills, and experiences. This 
program is one of the best experiences in school.” Some were able to see the connections 
between high school, work, and college: “It is a great experience and teaches you the work 
environment and a taste of the real world. It is also great for college and your resume.” 

Nearly all the internship supervisors praised the quality of students’ work, with several 
commenting that students had completed tasks at the level expected of some of their regular 
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The Career Academies Project 
Table 3.5 


Perceptions of Internship Program Quality as Reported by 
Interns and Supervisors, Pilot Academies 


Item 

Percentage 

Intern^ Derceotions 

Which of the following statements is true? 

You had a challenging job and enough work to do 

75.0 

You had a good relationship with your supervisor 

95.3 

You learned a lot 

89.3 

Overall, you liked your internship 

92.9 

Did the work you do benefit the organization? 

No, not at all 

4.8 

Yes, a little 

16.7 

Yes, some 

23.8 

Yes, a lot 

54.8 

How do you rate the internship seminars? 

Not useful 

1.2 

Somewhat useful 

39.5 

Very useful 

59.3 

The work in the seminars was: 

Too difficult 

0.0 

About right 

90.0 

Too easy 

10.0 

Total number of interns'* 

90 

SuDervisorS oerceotions 

Did the work performed by the intern benefit the organization? 

No, not at all 

0.0 

Yes, a little 

5.8 

Yes, some 

26.9 

Yes, a lot 

67.3 

Was the internship beneficial to the intern? 

No, not at all 

0.0 

Yes, a little 

0.0 

Yes, some 

17.3 

Yes, a lot 

82.7 

Would you consider sponsoring another intern in the future? 

92.5 

Total number of supervisors** 

53 


SOURCES: MDRC calculations are from the intern and employer surveys. 


NOTES: Responses are averaged across summer 2010 and summer 2011. The Academy of Art and 
Technology and the Center for Hospitality, Tourism, and Marketing started offering summer 
internships in summer 201 1, so these academies have a smaller weight in the averages. 

““Due to missing values on some survey items, the number of students and supervisors included in 
the analysis varies by measure. The total number of interns and supervisors reported in the table are 
the number of students or supervisors who responded to the surveys. Approximately 50 percent of all 
supervisors responded to the survey. 
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employees. In many instances, supervisors commented on the students’ growth in maturity and 
confidence. The opportunity to mentor a student was noted as a morale booster for employees. 
One reported that he and his coworkers had “renewed enthusiasm” after working with their 
intern for the summer. 

Box 3.4 describes a typical internship placement as experienced by students at one of 
the pilot academies. 


Resources: A Critical Factor in High-Fidelity Implementation 

The four academies that implemented the ECCO program during the pilot phase of the project 
did so with high fidelity to the model, resulting in an increased capacity to offer college and 
career exploration activities to more of their students. However, it is important to view this 
outcome in the context of the resources that were invested in program implementation. Of 
specific significance was the time spent by academy coordinators in planning and conducting 
ECCO activities. 

In its initial year of program implementation, each pilot academy received a sizable 
grant. In subsequent years, financial support was reduced but was nonetheless sufficient to 
compensate ECCO coordinators either through release time or a stipend. Coordinators invested 
significant time in organizing and managing ECCO. The most time-consuming activities related 
to participation in the training and technical support that was provided by the developer and to 
planning for and monitoring the internships. (Appendix Table B.2 presents detailed information 
about the number of hours per week and per semester that ECCO coordinators spent on various 
tasks in the pilot academies.) 

Also an important factor in the successful implementation of the program was the sup- 
port provided by the school districts. Each academy had a contact person — typically from the 
district office — who was responsible for career and technical education and on whom they 
relied for support. Sometimes they wanted assistance with negotiating district finances and 
policies; sometimes they needed help arranging for transportation to work sites and college 
campuses; and sometimes they sought expertise handling legal or administrative matters, such 
as arranging for credit for the internship seminar or for liability insurance and workers’ com- 
pensation insurance for the interns. Importantly, all four districts in the pilot study found 
funding to pay the seminar teachers. 


Looking Ahead 

At the end of school year 2011-2012, all four pilot academies expressed the desire to continue 
implementing the ECCO program. The academies had all integrated the explicit college and 
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Box 3.4 


A Typical Internship: 

The Digital Safari Academy Interns at John Muir Health Center 

Although the Digital Safari Academy is a multimedia company, the ECCO academy coordi- 
nators included in their search for internship hosts nonprofit organizations and other organi- 
zations that had small media departments and/or had media projects that were under way. 
This allowed interns to practice media skills in a context that interested them. Two students 
were placed at a local health care facility: One, with an interest in photography, worked as an 
assistant to the facility’s media specialist; the other was placed as an assistant in the media 
education department. 

The student who was assisting the media specialist learned to manage the online message 
boards for the doctors and helped create marketing materials for the hospital. Her tasks varied 
from day to day and included photography and photo editing — even a photo session during 
surgery. The student was proud that she had learned by observing and asking questions (even 
when she felt shy) and that, by the end of her internship, her supervisor could count on her to 
accomplish new assignments, even when he was out of the office. 

The student who was in the education department created logos and launch pages for the 
online workers’ training portal on a variety of topics, including latex safety, fire in the operat- 
ing room, and phlebotomy. In addition, she edited photographs for the institution’s archives. 
The student was proud of her role in educating the health center’s workers on safety proto- 
cols, and she felt that she had gained valuable practice in managing time and prioritizing 
tasks. She was also grateful for the professional mentoring that she had received from the 
education department’s three staff members, who supported her in improving her technical 
skills, gave feedback about her work, and talked with her at length about her plans for the 
future. Summing up the internship experience, she said: “As each day passed, 1 was learning 
more than 1 expected. 1 was able to learn new programs and was able to expand my creativi- 
ty, but most of all 1 learned how to communicate, ask questions, and develop not only a 
professional relationship but a personal bond.” She added that her supervisors were “so 
appreciative and never hesitated to thank me for all the work 1 did. It made me really happy, 
knowing 1 was doing a good job.” 


career preparation focus of ECCO into their academy programming; three of the academies had 
revised their Web sites and recruitment materials to incorporate language about the ECCO 
program and about the academy’s focus on preparing students for college and careers. As one 
coordinator stated: 

ECCO really changed what we do for the better. We are making more connec- 
tions between our curriculum and college, between our curriculum and career. 
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and between college and career. It’s more integrated. The students see it, and 
they are more motivated. 

Another coordinator said: 

We’ve been successful with ECCO because we [the teachers] are a cohesive 
team and we all understand the importance of ECCO and the impact it has. So 
we are all sharing in the teaching of workshops and are all willing to chaperone 
the career exploration and college visits. We are building this into the culture of 
our academy. 

Toward the end of the project, two of the pilot academies received recognition for ex- 
cellence, deriving in part from their participation in ECCO and their increased capacity to offer 
career and college exploration activities to their students. The Culinary Operations Academy 
was designated as a Model Academy by the National Career Academy Coalition, and the 
Academy of Art and Technology was named a Lighthouse Academy by the California State 
Department of Education’s California Partnership Academies initiative. 

Nonetheless, all four pilot academy leaders expressed concern about the extent to which 
they could continue the ECCO program once the grant period was over. While they believed 
that they could continue to teach the program’s lessons and could arrange career exploration 
visits and college visits, they were unsure about whether the school districts would cover 
transportation costs and find funding to compensate the ECCO coordinators during the summer 
internship program. 

The question of whether ECCO could be implemented equally well in academies with 
less support than that received by these four pilot academies spurred the expansion of the 
program into districts that were interested in implementing ECCO in several academies at the 
same time. This “scale-up phase’’ is described in Chapter 4. Then Chapter 5 returns to the pilot 
academies to examine student-level data, with the purpose of describing increases in participa- 
tion in the college and career exploration activities, greater awareness of college and career and 
the connections between them, and enhanced engagement in school. 
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Chapter 4 

The Implementation Experiences of the 
Scale-Up Districts and Academies 


Overview of the Scale-Up Phase 

As described in Chapter 3, four pilot academies implemented the Exploring Career and College 
Options (ECCO) program with a high degree of fidelity to its design, given significant financial 
assistance, training, and ongoing support from the curriculum developer. The extent to which 
the pilot academies’ success hinged on this support is an important question. If successful 
program implementation depends on significant financial and other resources, then the scalabil- 
ity and sustainability of a capacity-building intervention of this kind comes into question. 

The scale-up phase occurred during the third and final year of this project, in the 2011- 
2012 school year. Its primary goal was to assess the feasibility of scaling up ECCO in several 
academies within the same school district at the same time but with reduced support. If success- 
ful, this scale-up phase could inform the question of whether ECCO — or a program like it — 
is scalable and under what conditions. Three districts were chosen to implement the scale-up 
ECCO model in five career academies each: Oakland Unified School District (Oakland, 
California), Hillsborough County Public Schools (Tampa, Florida), and Miami-Dade County 
Public Schools (Miami, Florida). 

These districts provided three distinct settings, which helped to reveal more clearly the 
factors that promoted or impeded implementation of ECCO with reduced support. Oakland 
Unified School District was selected because it was participating in a statewide pathway 
initiative that emphasized work-based learning. Hillsborough County Public Schools was 
selected because it was home to one of the four pilot academies (the Culinary Operations 
Academy) and because district staff were interested in using the experience of this academy to 
support replication of the model in other academies. And Miami-Dade County Public Schools 
was selected because of the district’s commitment to supporting academies in general and work- 
based learning in particular; the district also had close ties with the National Academy Founda- 
tion (NAF), which assigned one of its employees in the Florida region to the role of supporting 
the academies that were chosen to implement ECCO. ' 


'This employee was contracted to work for only the academic year and, therefore, was not present during 
the internship program in the summer. The district played the lead role in supporting the academies during this 
phase of implementation. 
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While the ECCO model that was used by the scale-up academies was the same as the 
one implemented by the pilot academies, the level of support and the way that the support was 
delivered were modified. (Chapter 1, Figure 1.1, illustrates the ECCO model and theory of 
change.) Table 4.1 outlines the key differences in the types and levels of support given to the 
pilot and scale-up sites. First, the scale-up academies received smaller site payments from 
which to compensate the coordinators than the pilot academies received. Second, the technical 
assistance that had been provided directly to the four pilot academies by the program developers 
was provided instead by a district employee or employees and, in the case of Miami-Dade, in 
partnership between the district and the NAF employee. A “train-the-trainer” approach was 
used, whereby the program developer. Bloom Associates, trained the district staff (called 
“district facilitators”), who, in turn, provided direct assistance to the academies with support 
from Bloom Associates. The shift from a primarily developer-led operation to one that was 
primarily district-led with support from the program developer was less intensive and, therefore, 
potentially more scalable. Because the responsibility lay in the hands of the districts rather than 
the program developer, it was thought also that the model would be more sustainable. This 
chapter reports on the experiences of the scale-up academies in implementing ECCO under 
these conditions. 

In comparison with the pilot phase, data collection during the scale-up phase was also 
different. No student-level data were collected. Instead — in order to learn how the ECCO 
model was being implemented at the scale-up sites and to address the question of the feasibility 
of scaling up ECCO with less support — a combination of field visits, telephone interviews, and 
regular written reports (“implementation status reports”) was collected for each academy, and 
monthly field notes from the program developer were also reviewed. This information was 
assembled starting in summer 2011 and continuing through the end of summer 2012. (Appendix 
A describes all the quantitative and qualitative data sources used in this report.) 

The main finding from the scale-up phase of the project is that the academies were able 
to implement the core components of the ECCO program with reasonable fidelity and with less 
financial support and less technical assistance than the pilot sites received. With the exception 
of the three scale-up academies that withdrew from the project over the course of the year, all 
the others were able to implement the four ECCO components and their required activities 
(described in Chapter 2), including the internship program — which served almost the same 
proportion of the junior class as the pilot academies did. Whereas the pilot academies were able 
to teach all the required in-class lessons, the scale-up academies taught, on average, about 80 
percent of them. The scale-up academies also provided all the required off-campus visits to 
work sites and college campuses, with the single exception of one academy that did not conduct 
one of the two college visits. 
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The Career Academies Project 
Table 4,1 

Training, Support, and Site Payments in the Pilot and Scale-Up Sites 




Pilot Sites 


Scale-Up Sites 

Training Activity 

Year 1 

Year 2 

Year 3^ 

Year 1 

Phone check-ins 

Weekly with program 
developer 

Monthly with program 
developer 

1 per semester with 
program developer 

Monthly with district 
facilitator and occasionally 
with program developer 

Site visits with program developer 

1 per semester 

1 per semester 

1 per year 

0- 1 per semester 

In-person training 

2 per year 

None 

None 

2 per year 

Site payments'’ ($) 




4,000 per academy 

Academy of Art and Technology 

NA 

35,000 

20,000 


Center for Hospitality, Tourism, and Marketing 

NA 

35,000 

20,000 


Culinary Operations Academy 

35,000 

35,000 

10,000 


Digital Safari Academy 

35,000 

35,000 

10,000 



NOTES: “Only Digital Safari Academy and Culinary Operations Academy operated ECCO for a third year. 
•’Site payments were used primarily to pay for ECCO coordinators’ release time or stipends. 





The Scale-Up Academies: Recruitment and Attrition 

Recruitment of the scale-up academies took place in two phases. First, each district identified a 
group of academies that had either weak or nonexistent work-based learning programs. These 
academies were invited to participate in an early summer informational session provided by the 
program developer in partnership with the district. Of those that then volunteered to participate 
in the project, five academies were selected by each of the participating districts. Visits by the 
program developer or the district facilitators to each of the 15 academies began shortly thereaf- 
ter, to schedule the lessons and off-campus visits for the fall. 

As mentioned above, three academies withdrew from the project over the course of the 
year of the scale-up phase. One withdrew before implementation began. The other two with- 
drew several months later, well into implementation. Their reasons varied, but all three cited 
lack of time to carry out the activities and lessons as well as scheduling conflicts for their 
students. For the academy that withdrew before implementation began, no data were collected 
beyond the observations of the summer planning visits; this reduced from 15 to 14 the number 
of scale-up academies for which data were collected and are discussed below. 


The Implementation of ECCO’s Core Components 

The In-Class Lessons 

Participating academies at the scale-up sites were charged with implementing between 
13 and 18 ECCO lessons throughout the academic year, separately for the tenth- and eleventh- 
graders. Specifically, each academy was asked to teach: ^ 

• Four to five career exploration lessons for the tenth and eleventh grades in 
fall 2011 

• Four to seven career development lessons for the tenth grade and seven ca- 
reer development lessons for the eleventh grade in fall 2011 and/or spring 
2012 

• Four to five college exploration lessons for the tenth grade and six college 
exploration lessons for the eleventh grade in spring 2012 

As was the case for the pilot academies, the scale-up sites were not asked to teach the twelfth- 
grade curriculum during the first year of ECCO participation. 

^The numbers of lessons are given as ranges because some lessons were optional. 
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On average, the 14 participating scale-up academies taught approximately 80 percent of 
the ECCO tenth- and eleventh-grade lessons during the school year.^ Many academies found 
that fitting the lessons into the school schedule and course curriculum was challenging. About 
half the coordinators reported that they shortened some of the lessons to ease the pressure. The 
pilot academies also modified lessons to shorten them. In most academies, all the students in the 
tenth and eleventh grades who attended school on the days that the lessons were taught partici- 
pated in the lessons. This did not occur in a few academies where the entire grade was not 
grouped together in at least one class (called “cohorting”). In these academies, only a subset of 
students participated in the lessons, and only those who were prepared in this way were allowed 
to participate in the career exploration and college visits. One of these academies eventually 
dropped out of the program because of this scheduling issue, underscoring how important 
cohorting is for full implementation of the program. 

As was the case for the pilot academies, the biggest challenge that ECCO coordinators 
in scale-up academies faced in teaching the lessons was finding the time to teach them. One 
coordinator who taught in a school where a state improvement program was under way said that 
the strict pacing requirements that the school was charged with implementing made it extremely 
difficult to find time to teach the ECCO curriculum: 

We are held to different mandates. We have state and district coming in to ob- 
serve teachers on a weekly basis and an instmctional review on a quarterly basis. 

Those are some things that make this challenging overall. [ECCO] provides an 
excellent curriculum, but when we’re held by pacing guides, preassessment data, 
and end-of-course exams, all of that assessment data (pre, mid, end) — it makes 
it very challenging to be flexible about incorporating other curricula. 

Despite the scheduling challenges, as mentioned above, most of the scale-up academies 
taught most of the lessons, and the ECCO coordinators and district facilitators reported that the 
lessons significantly enhanced the learning that occurred on the career exploration visits and 
college visits. Table 4.2 shows that, during the fall telephone interviews, 100 percent of ECCO 
coordinators indicated that the ECCO lessons were “somewhat engaging” or “very engaging” 
for their students. As one coordinator put it: 

What 1 and the students liked is the workshops [lessons]. We’ve been on the ca- 
reer exploration visits, but with this curriculum there was a focus from beginning 
to end. Other than the lessons, 1 also really liked the questions they had to pre- 
pare, the research they had to do before the visit, and the reflection. Those three 
components made a huge difference. 


^Although coordinators were interviewed near the end of each semester to capture the bulk of activities 
that were implemented, it is possible that a few lessons were implemented after the collection of these reports 
and are not reflected in the data. 
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The Career Academies Project 
Table 4.2 


ECCO Coordinator^ Views on Lessons and Career Exploration Visits, 

Scale-Up Academies 


Item 

Number Percentage 

Views about lessons 

How challenging was fitting your lessons into your regular curriculum this semester? 

Not at all challenging 

1 

7.1 

Somewhat challenging 

7 

50.0 

Very challenging 

5 

35.7 

Missing 

1 

7.1 

Have you significantly modified any of the lessons? 

No 

6 

42.9 

Yes 

7 

50.0 

No response 

1 

7.1 

How engaging were the ECCO lessons for your students? 

Not at all engaging 

0 

0.0 

Somewhat engaging 

10 

71.4 

Very engaging 

4 

28.6 

Views about career exuloration visits 

Were the visits successful in achieving their goals? 

Not at all successful 

0 

0.0 

Somewhat successful 

2 

14.3 

Very successful 

12 

85.7 

Are you providing more work-based learning opportunities to more of 
your students than last year? 

Fewer students than last year 

2 

14.3 

About the same number of students 

3 

21.4 

More students than last year 

9 

64.3 

Are the work-based learning opportunities of higher quality than similar experiences 
you may have offered last year? 

Poorer quality 

2 

14.3 

Somewhat better quality 

3 

21.4 

Much better quality 

7 

50.0 

Did not offer similar experience last year 

0 

0.0 


T otal number of coordinators 1 4 


SOURCE: MDRC calculations are based on fall 2011 phone interviews with ECCO site coordinators. All 14 
coordinators responded to the questions. 
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Similarly, one of the district facilitators indicated that, 

1 think the prep lessons ahead of time, and the lessons after, provided a tremen- 
dous amount of stmcture and scaffolding and accessibility to the experience that 
teachers had not provided prior to the [ECCO] curriculum. 

The Career Exploration Visit 

As described in Chapter 2, the ECCO career exploration visit is a half-day, off-site visit 
to a company or organization whose work is aligned with the academy’s career theme. The 
purpose of the visit is to provide students with an opportunity to learn more about the workplace 
and to interact with people who work in their career theme area. 

Scale-up academies were required to conduct one visit for the tenth-graders and one for 
the eleventh-graders each fall. If the number of students exceeded 30 or so, the coordinators had 
to arrange several visits on the same or different dates to accommodate all the students while 
maintaining the personal feel and hands-on character of the experience. By the end of the year, 
all the scale-up academies conducted all the required visits. Moreover, of the 14 coordinators, 
10 reported that students learned more from these visits than they had on previous field trips to 
work sites, before ECCO. One coordinator noted: “Students really were engaged in seeing 
adults working and thinking about their own careers.” 

Several factors played into the successful implementation of the career exploration vis- 
its in academies with fewer resources and coaching than the pilot academies had. First, employ- 
er hosts were generally well prepared for the visit. While not all coordinators were able to have 
an in-depth conversation with the employer host ahead of time — usually due to time con- 
straints — those who did felt that they were able to provide a better experience for their stu- 
dents. A guide for employer hosts, developed for the ECCO program, was perceived as a useful 
way to communicate with hosts — even when the coordinator did not have time to do so in 
person, as was the case for coordinators at many scale-up academies. As one coordinator said 
about the guide: “It gave [employer hosts] an idea of what our purpose was and what we wanted 
the kids to learn. That’s where it really made a difference.” 

All the ECCO coordinators who participated in the 2011 telephone interviews indicated 
that the work site visits were “somewhat successful” or “very successful” in achieving their 
goals (Table 4.2)."^ More than half the coordinators indicated that more students participated in 


"^The goals of the career exploration visits are to (1) expand students’ awareness of careers and work envi- 
ronments; (2) make connections between what is learned at school and what is expected in the workplace; (3) 
observe how employees apply job-specific skills, how transferable skills are used in a variety of jobs, and how 
technology is used in the workplace; and (4) develop the skills to interact comfortably and confidently with 
working adults. 
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visits to workplaces than before the ECCO program, and 10 of the 14 felt that the experiences 
were of “somewhat better” or “much better” quality than any that had been provided previously. 

The College Visits 

As described in Chapter 2, the ECCO model includes at least two college visits each 
year — one for tenth-graders and another for eleventh-graders. Across the three scale-up school 
districts, 13 of the 14 academies conducted all the college visits as planned. 

Compared with the visits to work sites, the college visits appear to have been easier to 
implement for some ECCO coordinators in the scale-up academies because many of them 
already had connections to a postsecondary institution. As was the case for the pilot sites, when 
the scale-up sites did experience challenges, it was typically because the college visits were 
postponed until late spring, rather than being scheduled earlier in the year as recommended by 
the program developer. 

The Internship Program 

The scale-up academies and school districts viewed the ECCO internship and seminar 
component as the capstone experience for their students. As Chapter 3 describes, the four pilot 
academies were successful in creating internships that followed the ECCO model for most if not 
all of their interested and available students. Surprisingly, however, many students were not 
available to participate due to competing demands on their time, such as summer school or 
summer jobs. 

The experience of the scale-up sites was similar to that of the pilot academies. Figure 
4.1 shows the intake and participation rates for student interns across the scale-up districts. 
ECCO coordinators in the 10 academies that provided data identified 423 eleventh-graders who 
had participated in enough of the ECCO activities to be prepared for an internship. Of those, 
174 (41 percent) were available to participate in an internship and opted in. Two students were 
not placed due to lack of an employer host, and, of the remaining 172 students, 137 (80 percent) 
were placed in an internship. The scale-up academies varied considerably in the percentage 
of students who participated in an internship. 

The coordinators reported that the main reason why students were not available for in- 
ternships was not a lack of employer hosts. As was the case for the pilot academies, only a 
handful of students in scale-up sites were not placed for that reason; in fact, several coordinators 
reported that they had more employer hosts than they had students available to participate. Nor 
was there a lack of student interest in internships. Many students could not participate because 
they were behind in their course credits and were required to attend credit-recovery programs in 
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The Career Academies Project 


Figure 4.1 

Student Intake and Participation in the Internship Program: 
Scale-Up Districts, 2012 



SOURCES: MDRC calculations are based on ECCO summer implementation status reports and validated by the program 
developers. 

NOTE: “This ineludes all reasons that a student did not participate, ineluding the need to attend credit recovery programs, 
earn money in summer jobs, and transportation costs. 
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summer school. Others already had jobs lined up. Transportation was another barrier; some 
students either could not afford the bus fare to commute to their internship sites or to the 
school where the seminar was held or had parents who were reluctant to let them use public 
transportation. 

The academies in all three scale-up districts taught the six-week internship seminar, alt- 
hough a few had to reduce the number of lessons that they taught because they ran out of time. 
Interestingly, in one district, none of the five coordinators taught the seminar — a modification 
of the model, which calls for the seminar to be taught by an academy coordinator who knows 
the students. Instead, the district hired other teachers to teach it, primarily because none of the 
coordinators were available to work in the summer. 

Also like the pilot academies, the scale-up academies reported positive experiences for 
the students who participated in the internships and attributed these to the high level of prepara- 
tion of both the students and the employers. In response to a question about what made the 
internship program work as well as it did, one coordinator said: 

Having the confidence that the kids were really ready for a work site experience. 

1 think it built up kids’ confidence; it impressed employers. It was being able to 
sell to host sites that if they took an [ECCO] intern, it was an intern that was 
coming with a supervisor, that knew about 21st-century skills, and that they were 
prepped ahead of time in a systematic and sequenced way. 

Another coordinator highlighted the reactions of her students: 

[Students] were very engaged. They were really excited about their final presen- 
tations. A lot of them weren’t expecting that they would get all that they got out 
ifom the internships — that they would interact with adults, be able to perform 
well; everything that they’ve learned over the past three years, that they would 
actually be able to put that into practice. 


Training and Support 

As mentioned above, the academies that implemented ECCO during the scale-up phase were 
supported with a train-the-trainer model, which involved much less direct contact with the 
program developer than was the case for the pilot academies. As designed, the train-the-trainer 
model called for the program developer to model each step of the process while the district 
facilitator observed. Next, the district facilitator took the lead, and the program developer 


56 



observed and provided support as needed. Subsequent meetings were led by the district facilita- 
tor alone. ^ 

All the ECCO coordinators and a few additional academy teachers in the scale-up sites 
participated in the two trainings provided by the developer over the course of the year. While 
coordinators and facilitators generally found the in-person trainings to be helpful, some said that 
a half day rather than a full day would have been adequate to learn the material. 

Coordinators also found the monthly telephone meetings with the program developer 
and/or the district facilitator to be adequate. As one coordinator put it: “The monthly phone calls 
are great and keeps us on our game. It keeps us pointed in the right direction.” Similarly, a 
coordinator said: “Just keeping me on task, the professional development [monthly calls] we 
have once a month, the e-mail reminders keep us really focused and carry their weight in gold. 
They’ve been the stabilizer.” 

Some coordinators attributed much of their success in implementing the program to the 
support of their district facilitators. “Having a direct contact from the district was really helpful 
. . . they really motivated me, sent me reminder e-mails. That, along with the materials with 
templates, was really helpful. Doing it all on our own would’ve been impossible.” 


Conclusions 

After only one year — and with substantially fewer resources and less support from the pro- 
gram developer and more from the school district than the pilot academies received — the 
academies in all three scale-up districts succeeded in implementing the ECCO program. They 
offered their tenth- and-eleventh graders a set of well-implemented career and college curricu- 
lum and exploration activities, along with an internship program that, according to them, 
exceeded what they had offered in the past in terms of both quality and the number of students 
served. Compared with the achievements of the pilot academies, the scale-up academies fell 
only slightly short of accomplishing full implementation. A few of them did not teach the full 
set of lessons, and a few did not include all the students in the college visits and workplace 
visits. But all the career exploration visits took place as planned, as did all but one of the college 
visits. The percentage of interested and available students who were placed in internships is 
comparable to the result in the pilot academies. The study identified several factors that appear 
to have facilitated implementation of ECCO at scale. 


^While program implementation in the scale-up sites generally followed this procedure, in practice, the 
district facilitators sometimes needed more support from the program developers than anticipated. In these 
cases, the program developer worked directly with the academies more often. 
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First, as with many district-led initiatives, it helped to have strong, well-respected dis- 
trict employees who could promote, protect, and support the program. Of particular value was 
district assistance in locating potential employer hosts for the career exploration visits and 
internships. Whether it was a list of employers organized by industry, a career fair attended by 
employers and students, or an offer to make the first contacts with potential employer partners, 
some academies depended on help from the district. 

Second, alignment of the program’s goals with those of other district initiatives and pri- 
orities — particularly, work-based learning programs — promoted smoother implementation. In 
Oakland Unified School District, the broad support from the mayor for rolling out internships 
throughout the district paved the way for ECCO to help the scale-up academies meet these 
objectives. In Miami-Dade County Schools, where the district had operated an internship 
program for many years, ECCO was viewed as a valuable enhancement of an already-strong 
work-based learning program for some students. While differences between the ECCO intern- 
ship program and the district’s internship program initially caused some confusion among 
students and internship hosts, this was quickly addressed and resolved. 

Third, it was essential to have sufficient commitment at the academy level to take on 
the work of implementing the program. Three scale-up academies withdrew from the project, in 
large part because they were not ready or not fully committed to invest the time and effort in the 
program. Academies appeared to be more motivated when their participation was tmly volun- 
tary and when they viewed ECCO as a clear benefit for their students rather than just one more 
project that the district was promoting. 

Fourth, some form of compensation for the ECCO coordinators appears to be needed as 
an acknowledgement of the extra time required to implement the program. While the pilot 
academies generally used their grants to buy release time for the coordinators, the scale-up sites 
received a smaller amount and generally paid it to the coordinators in the form of a stipend. 
Although the stipend was not enough to be considered compensation for the number of hours 
that they put into the project, it seems to have helped motivate the coordinators to spend their 
afternoons and evenings contacting employers. 
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Chapter 5 

Exploring the Potential of ECCO for 
Improving Short-Term Student Outcomes 


Introduction 

The purposes of this project were, first, to develop and pilot a new program — complete with 
curricula and guides — that high school career academies and similar programs can adopt to 
build career and college exploration programs and, second, to assess the promise of that 
program in improving the short-term outcomes predicted by its theory of change (Chapter 1, 
Figure 1.1). The most direct outcome of interest was whether Exploring Career and College 
Options (ECCO), a capacity-building program, actually resulted in increased capacity of career 
academies to offer career and college exploration opportunities to their students and to improve 
the quality of those experiences. 

The preceding two chapters offer evidence based on mostly qualitative data to support 
the conclusion that, on the whole, both the pilot academies (Chapter 3) and the scale-up acade- 
mies (Chapter 4) were able to offer their students a range of career and college exploration 
experiences that they previously had not offered and that, according to their teachers, students 
seemed to learn more from them than in the past. This chapter explores that theme further using 
quantitative data: Are there indications that students participated in college and career explora- 
tion activities at higher rates, and — possibly as a result of increased exposure to these experi- 
ences — are there any signs of changes in such short-term outcomes as students’ awareness of 
postsecondary options and improved 21st-century skills? 

The results reported here are not intended to demonstrate whether there is — or is not 
— a causal link between the ECCO program and changes in student outcomes. That inference 
would require a more rigorous design, such as a randomized controlled trial, that ensures that 
the students who received the program services had the same characteristics before the program 
as the students who did not receive the services. Only through such a design can one isolate the 
effect of the program on student outcomes. 


Research Questions and Methods 

ECCO was designed to build the capacity of career academies to offer activities and experienc- 
es to their students that would expand their awareness of college and career options, help them 
see the connection between school and their futures, improve their 21st-century skills, and 
deepen their engagement in school. While improvements in these short-term outcomes might. 
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after high school, lead to such outcomes as increased enrollment in college, a successful 
transition to the workplace, and higher earnings, these longer-term outcomes were not studied 
as part of this project. Instead, the focus is on the shorter-term outcomes, that is, those that may 
come about during or soon after participation in the ECCO activities. Appendix Table C.l lists 
the constmcts underlying the short-term outcomes measured for this study, along with their 
definitions and the survey items that were used to create scales measuring growth. 

This chapter assesses changes in four short-term student outcomes: 

• Did students in academies that offered ECCO participate in more college 
and career exploration activities and have more opportunities to learn 21st- 
century skills than they would have without the ECCO program? 

• Did students in academies that offered ECCO show more growth in their 
awareness of career and college options and in 21st-century skills than they 
would have without the ECCO program? 

• Did students in academies that offered ECCO have a better understanding of 
the connections between what they were learning in high school and their fu- 
tures than they would have without the ECCO program? 

• Were students in academies that offered ECCO more engaged in school than 
they would have been without the ECCO program? 

The analyses use a before-and-after (pre-post) design to assess ECCO’s potential for 
improving student outcomes. Outcomes for consecutive grade cohorts are compared for 
different levels of exposure to the program. For example, to measure whether ECCO is associ- 
ated with stronger awareness of the connection between high school and the student’s future, 
the percentage of tenth-graders who reported understanding that connection in the year before 
their academy implemented ECCO is compared with the percentage of tenth-graders who 
reported understanding that connection a year later, after their academy had adopted the 
program. 

To assess changes in outcomes, three comparisons are made, each corresponding to 
different numbers of semesters of “exposure” to the ECCO program. Differences in out- 
comes for grade cohorts are compared at: 

• Zero and two semesters of exposure (change between no participation and 
one year of participation) 

• Two and four semesters of exposure (change between one year and two years 
of participation) 
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Zero and four semesters of exposure (change between no participation and 
two years of participation) 


A Note About the Comparisons 

Because the four pilot academies began ECCO implementation at two different points in 
time, the three comparisons above are measured for different academies. The first comparison 
(between no exposure and two semesters of exposure) uses data on the students who were 
enrolled in the Academy of Arts and Technology and in the Center for Hospitality, Tourism, and 
Marketing. The second and third comparisons use data on the students who were enrolled in the 
Digital Safari Academy and in the Culinary Operations Academy. The first pair of academies is 
referred to in this chapter as “first-year academies,” and the second pair is referred to as “second- 
year academies.” Box 5.1 further explains the terminology used throughout this chapter. 

Comparing grade cohorts (for example, a cohort of tenth-graders with zero semesters of 
participation and the next year’s cohort of tenth-graders with two semesters of participation) is 
by itself not sufficient to conclude that ECCO contributed to any observed growth. For exam- 
ple, growth could be due to preexisting differences in the characteristics of the cohorts being 
compared. For this reason, individual-level change is also assessed, when possible, and is 
reported in Appendix C. In this approach, change in outcomes is compared for the same 
students at two points in time, corresponding to different levels of exposure to the program. If 
change is observed at the individual level in addition to change at the cohort level, confidence 
increases that the change can be attributed at least partly to ECCO. 

A Note About Attrition 

Some students who were in an academy when ECCO was first implemented were no 
longer enrolled in the academy a year later. As Figure 5.1 shows, a large percentage (about one- 
third) of the tenth-graders in 2010 were no longer enrolled in the academy a year later. Coordi- 
nators were asked why each student left, and the results are shown in the pie chart at the bottom 
of the figure. For the 109 students who left the academies before graduation and for whom 
coordinators knew the reason for leaving, nearly half (45 percent) did so because they left the 
high school in which the academy was located, either dropping out altogether or moving to 
another high school. Another 29 percent stayed in the high school but withdrew from the 
academy. A small number of students left the academy because they were no longer interested 
in the academy theme (4 percent), because of academic reasons (7 percent), or because of 
discipline issues (3 percent). 

It is likely that students who left the academy differed from students who stayed in the 
academy. How this affects the results reported in this chapter is unknown, but it is reasonable to 
assume that students who left were more tenuously connected with the academy or the high 
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Box 5.1 


A Strategy for Assessing the Promise of ECCO 
to Improve Student Outcomes 

The following questions and answers (“Q&A”) are offered to clarify the text discussion. For the 
technical aspects of the approach used to assess changes in student outcomes, see Appendix C. 

Which pilot academies are “first year” and which are “second year,” and what is the differ- 
ence? The Academy of Arts and Technology and the Center for Hospitality, Tourism, and Mar- 
keting began implementing ECCO one full year after the Culinary Operations Academy and the 
Digital Safari Academy. Students in the former two academies thus participated in ECCO for a 
shorter time than students in the latter two. For this reason, the first two are referred to as the 
“first-year academies,” and the second two are referred to as the “second-year academies.” 

At what points were student outcomes measured? Most outcomes were measured using a 
student survey that was administered twice: in April 2010 and in April 2011. All students who were 
in school on the day of the survey in all four pilot academies were asked to fill out a questionnaire. 
The survey included questions that were later used to calculate the percentages of students who had 
participated in various college and career exploration activities, as well as their scores on the four 
scales (career awareness, college awareness, 21st-century skills, and engagement). 

Was there a baseline? At the time of the first survey (April 2010), the first-year academies had 
not yet implemented the ECCO program — they would do so in the fall of that year. The survey 
in these academies, therefore, provided true “baseline” measures for the outcomes. The second- 
year academies had already operated ECCO for two semesters (fall 2009 and spring 2010) at 
the time of the first survey, so that the survey provided measures of outcomes after two semes- 
ters of exposure. At the time of the second survey (April 2011), the first-year academy students 
had had two semesters of exposure to the program, and the second-year academy students had 
had four semesters. 

Which grades received the ECCO program and for how long? 

First-year academies. Students in both tenth and eleventh grade at the time of the April 2010 
survey had not yet participated in ECCO. These students had been exposed to ECCO for two 
semesters by the time of the April 2011 survey. A twelfth-grader at the time of the first survey had 
no exposure to ECCO at either the first or the second survey. 

Second-year academies. Students in both tenth and eleventh grade at the time of the April 
2010 survey had two semesters of exposure to ECCO and had four semesters by the time of the 
second survey. A twelfth-grader had no exposure to ECCO by the first survey, because the initial 
year of implementation did not include twelfth grade, but twelfth-graders in these academies at the 
time of the second survey had four semesters of ECCO. 

What do the exhibits show? The exhibits in Chapter 5 generally make one of three comparisons, 
each capturing changes in outcomes over time. The first comparison below relies on the first-year 
academies, and the second and third comparisons rely on the second-year academies: 

1 . Outcomes measured at zero and two semesters of participation in ECCO 

2. Outcomes measured at two and four semesters of participation in ECCO 

3. Outcomes measured at zero and four semesters of participation in ECCO 
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Percentage enrolled 


The Career Aeademies Projeet 
Figure 5.1 

Retention Rates and Reasons for Leaving Career Academies 


Percentage of Tenth-Grade Stndents Enrolled in Academies in Spring 2010 
Who Were Still Enrolled in 2010-2011 and 2011-2012 (N = 277) 



Reasons for Leaving Academy After Spring 2010 or Spring 2011 
Stndents in 10th and 11th grades (N = 161) 


Other Withdrew from 



Left high school 

45% 

SOURCES: MDRC calculations are based on the student withdrawal tracking sheet and academy class 


n) 


school. In tenns of observable characteristics, students who were present in the academies in 
spring 2011 to take the survey but not in 2010 were similar. (See Appendix Table C.3.) 


Findings 

Changes in Rates of Participation in Career Expioration Activities and in 
Coiiege Expioration Activities 

This section presents findings from analyses using survey data to discover whether stu- 
dents’ participation in a sample of career and college exploration activities and experiences 
increased as a result of the increased capacity of the pilot academies to offer these activities. The 
goal is to determine whether any growth in participation in such activities can plausibly be 
attributed to ECCO and whether growth in the first year was sustained into the second year of 
the program. 

Career Exploration Activities 

Figure 5.2 shows the percentages of students in the tenth and eleventh grades who re- 
ported having participated in exploration activities with no semesters of ECCO (the light bars) 
compared with tenth- and eleventh-graders with two semesters of ECCO (the dark bars). 
Asterisks indicate whether the differences are statistically significant. The figure shows signifi- 
cant growth in participation rates for all indicators but one: whether the student reported having 
had opportunities to practice or learn 21st-century skills. 

For many indicators, the before-and-after differences in participation rates are substantial 
as well as statistically significant. For example, the percentage of the students who reported 
having done a practice job interview doubled, growing to 52 percent after two semesters of 
ECCO. The largest gain was in the percentage of students who reported having visited a work 
site. Just 11 percent of the pre-ECCO students said they had done so in the 2010 survey, com- 
pared with 52 percent in the 2011 survey. This pattern of results is consistent with an analysis of 
change in participation rates for the same students over time, strengthening the evidence that 
ECCO was a cause of increased participation in these activities (Appendix Table C.4). ' 


*The one exception is the indicator of whether students had learned “what level of education is needed for 
various careers.” As shown in Appendix Table C.4, there was no corresponding difference in the analysis of 
individual-level growth in this activity, casting some doubt on the possibility that the cohort-level growth seen 
in Figure 5.2 can be attributed to ECCO. 
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The Career Academies Project 
Figure 5.2 

Participation in Career Exploration Activities During the School Year: 

Comparison of Tenth- and Eleventh-Grade Cohorts Combined, with Zero and Two Semesters of ECCO. 

First-Year Academies (AO AT and HTM) 


□ Spring 2010 (N = 280) □ Spring 20 1 0 (N = 2 74) 


Attended guest speaker presentation about jobs or workplaee skills 
Learned in elass about how to write a good resume 
Did a praetiee job interview 

Learned in elass about how to act appropriately in the workplace 

Learned in class about how to respond to workplace challenges like 
discrimination/harassment 

Had opportunities to practice or learn 2 1 st-century skills 
Discussed with a teacher or counselor what career would be best for you 
Worked with a teacher or counselor to create a plan for your future 
Learned what level of education is needed for various careers 
Learned what level of education is needed for a career that interests you 
Visited a work site as part of a school activity 
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Figure 5.2 (continued) 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on survey respondents enrolled in a given grade in spring 2010 and spring 
2011. The white bars are the regression-adjusted outcomes of students who responded to the spring 2010 
student survey. The dark bars are the regression-adjusted outcomes of students who responded to the spring 
2011 survey. 

Outcomes are adjusted for the following student characteristics: academy fixed effects, grade fixed effects, 
whether a student belonged to a racial or ethnic minority, gender, mother's educational attainment, whether a 
student was ever held back, whether the student received a D or below in English or math in the previous 
school year, and missing indicators for each of these characteristics. 

A two-tailed t-test is used for all statistical tests presented in this figure. Statistical significance levels are 
indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Due to missing values on some survey items, the number of students included in the analysis varies by 
activity. The numbers of students reported in the figure are the number of students who responded to the 
survey. 

AO AT = Academy of Art and Technology; HTM = Center for Hospitalty, Tourism, and Marketing. 


Students with two semesters of exposure to ECCO activities were no more likely to re- 
port that they had opportunities to learn about 21st-century skills than students with no semes- 
ters of ECCO. On the other hand, a large percentage of students (70 percent to 80 percent) 
reported having had such opportunities both before and after their academies implemented the 
program, so room for improvement was limited. 

The participation story is a bit different when comparing grade cohorts who had two 
semesters of ECCO with those who had four semesters of ECCO. ^ As Appendix Figure C.l 
shows, the rate of participation in these activities was already high for most activities by the end 
of spring 2010. In fact, the participation rate for most activities was higher than it was for the 
other two academies after the same number of semesters. There was little difference in the 
participation rates between two and four semesters of ECCO. That is, for the most part, about 
the same percentage of grade cohorts reported participating in these activities each year, 
suggesting that the program was implemented with about the same degree of fidelity in both 
years and that having a second year of exposure to the program did not increase participation 
rates. This conclusion is generally supported by the analysis of change in participation for the 
same individuals over time (Appendix Tables C.5 and C.6). 

Finally, Appendix Figure C.2 compares the participation rates for the twelfth-grade 
cohort with no semesters of ECCO and the twelfth-grade cohort with four semesters of 


^Only the exhibits showing comparisons in the participation rates between students with zero and two 
semesters of ECCO are included in this chapter. Appendix C presents results for all other comparisons (that is, 
between two and four semesters of participation and between zero and four semesters of participation). 
Findings are summarized in the chapter. 
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ECCO. The most striking difference is the percentage of the students who reported having 
had an internship. The percentage increased from 19 percent in 2010 to 47 percent in 2011. 
Participation increased for several other indicators, including visiting a work site, learning 
about what level of education is needed for various careers, doing practice interviews, and 
preparing a resume. For most of the activities, however, there was no statistically significant 
growth in participation. 

College Exploration Activities 

Figure 5.3 shows that the rates of participation in activities designed to increase college 
awareness are substantially and significantly higher for the grade cohort with two semesters of 
ECCO than for the same-grade cohort before ECCO was implemented. Significant growth is 
seen in the percentage of students who reported having participated in college exploration 
activities, such as talking with a teacher or other adult at school about college, visiting college 
campuses, looking up information about colleges, and attending college fairs. Analysis of 
change in participation rates in college exploration activities for the same individuals over time 
supports the interpretation that ECCO was a cause of this growth, at least in part (Appendix 
Tables C.7 and C.8). Appendix Figure C.3 shows that — like the participation rates in career 
exploration activities — the participation rates in college exploration activities are roughly equal 
for the grade cohorts with two and four semesters of ECCO. Appendix Figure C.4 shows, 
however, that the twelfth-grade cohort with four semesters of ECCO reported higher rates of 
participation than the twelfth-grade cohort with no semesters of ECCO, for several college 
exploration activities. The students with two years of ECCO were substantially and significantly 
more likely to attend a college fair, attend a presentation with college representatives, and visit a 
four-year campus than the earlier cohort of students with no semesters of ECCO. A similar 
pattern is evident in the analysis of participation for the same group of students over two 
semesters, strengthening confidence in the evidence that ECCO was at least one of the reasons 
for this growth (Appendix Table C.9). 

The overall trend is consistent with the implementation study reported in Chapter 3: 
The academies were successful in implementing the career and college lessons and visits, and 
so students were more likely to participate in these activities. 

Changes in Short-Term Outcomes 

Results of the analysis of short-term outcomes are presented next. As in the analysis of 
participation rates, the results are presented separately for the first-year academies and for the 
second-year academies. The analysis compares the outcomes of same-grade cohorts in order to 
measure growth between zero and two semesters, between two and four semesters, and between 
zero and four semesters of ECCO participation. 


67 



The Career Academies Project 
Figure 5.3 

Participation in College Exploration Activities During the School Year: 

Comparison of Tenth- and Eleventh-Grade Cohorts Combined, with Zero and Two Semesters of ECCO, 

Eirst-Year Academies (AOAT and HTM) 


□ Spring 20 1 0 (N = 280) ■ Spring 20 1 0 (N = 274) 


Talked with a teacher or other adult at school about college 
Talked with a parent or other family member about college 

Attended a college fair 
Attended a presentation by college representatives 
Looked up information about colleges on the Internet 
Visited a 4-year college campus 
Visited a community college or career-technical school campus 
Talked with an adult about how to pay for college 
Took the P-SAT (or signed up to take it this summer) 
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Figure 5,3 (continued) 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on survey respondents enrolled in a given grade in spring 2010 and spring 2011. The white bars are the regression- 
adjusted outcomes of students who responded to the spring 2010 student survey. The dark bars are the regression-adjusted outcomes of students 
who responded to the spring 2011 survey. 

Outcomes are adjusted for the following student characteristics: academy fixed effects, grade fixed effects, whether a student belonged to a 
racial or ethnic minority, gender, mother's educational attainment, whether a student was ever held back, whether the student received a D or 
below in English or math in the previous school year, and missing indicators for each of these characteristics. 

A two-tailed t-test is used for all statistical tests presented in this figure. Statistical significance levels are indicated as follows: *** = 1 percent; 
** = 5 percent; * = 10 percent. 

Due to missing values on some survey items, the number of students included in the analysis varies by activity. The numbers of students 
reported in the figure are the number of students who responded to the survey. 

AO AT = Academy of Art and Technology; HTM = Center for Hospitalty, Tourism, and Marketing. 
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Table 5.1 presents changes in the outcomes measured at zero semesters and at two se- 
mesters of participation in ECCO. There are statistically significant differences between the 
scores for the same-grade cohorts with zero and two semesters of ECCO for both career 
awareness and college awareness. The pattern of change for the same individuals over time 
generally supports the finding that growth occurred at least in part due to ECCO, because 
statistically significant differences between scores at zero and two semesters are observed for 
both career awareness and college awareness (Appendix Table C.IO). There is no comparable 
evidence that ECCO had an effect on either engagement or 21st-century skills. 


The Career Aeademies Project 
Table 5.1 

Changes in Average Scores on Scales Measuring Short-Term Outcomes 
from Spring 2010 to Spring 2011, First-Year Academies (AOAT and HTM) 


Outcome 

Spring 2010 

Spring 2011 

Estimated 

Change 

Effect 

Size 

P -Value 
for Change 

10th and 1 1th grades 
Semesters in ECCO program 
Engagement 

0 semesters 

2.91 

2 semesters 
2.93 

0.01 

0.03 

0.753 

Career awareness 

2.94 

3.09 

0.16 

0.20 

0.022 ** 

College awareness 

2.72 

2.94 

0.22 

0.28 

0.002 *** 

Self-assessed 2 1 st-century skills 

3.28 

3.34 

0.06 

0.12 

0.234 

Total number of students^ 

280 

274 





SOURCE; MDRC calculations are from the ECCO spring student surveys. 

NOTES; The student outcomes presented in this table are calculated by averaging across a student's responses 
to relevant survey items. These survey items (and the resulting outcomes) are on a 4-point scale (1 = not at all 
true; 2 = not very true; 3 = sort of true; 4 = very true). This analysis is based on students who responded to both 
the spring 2010 and 2011 student surveys. The "Spring 2010" column presents the outcomes for these students 
in spring 2010. The "Spring 2011" column presents the outcomes for these students in spring 2011. The 
"Estimated change" column is the change in outcomes for these students from spring 2010 to spring 
2011. Hypothesis testing accounts for repeated observations of students across survey years. 

A two-tailed t-test is used for all statistical tests presented in this table. Effect sizes are based on the standard 
deviation of an outcome for all survey respondents in spring 2010 and spring 2011. Statistical significance 
levels are indicated as follows; *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

'‘Due to missing values on some survey items, the number of students included in the analysis varies by 
outcome. The total number of students reported in the table are the number of students who responded to the 
survey. 

AOAT = Academy of Art and Technology; HTM = Center for Hospitality, Tourism, and Marketing. 
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Table 5.2 presents changes in the outcomes measured at zero and two semesters and at 
zero and four semesters of participation in ECCO. There are no significant differences in scores 
between same-grade cohorts with two and four semesters of ECCO. However, the change in 
outcomes measured at zero and four semesters is significant for career awareness and is close to 
statistical significance for college awareness.^ No difference is evident for the other outcomes. 

Finally, an analysis was conducted of students’ responses to two statements that were in- 
tended to measure their beliefs about the connections between school and their future — a connec- 
tion that ECCO was designed to promote. As shown in Table 5.3, the percentages of students who 
indicated that they believed that there is a connection between school and what they will do after 
high school are significantly greater for students with two semesters of ECCO than for the same- 
grade cohort a year earlier, before ECCO was implemented. Analysis of change in this outcome for 
the same students over time also suggests that ECCO was a likely influence on growth (Appendix 
Table C.12). However, no significant difference is seen in students’ responses to the statement 
“What is learned in school is important for later in life.’’ Nor, as Appendix Table C.2 shows, are 
there statistically significant differences for these two measures between the students with two and 
four semesters of ECCO or between the students with zero and four semesters. 

To sum up the findings from this analysis of short-term outcomes, it seems plausible 
that ECCO may have led to growth in career awareness and college awareness during the first 
two semesters of participation, when students were in tenth and eleventh grades. For most 
indicators, however, there is insufficient evidence that a second year of exposure to ECCO 
resulted in growth above and beyond the growth that occurred in the first two semesters of 
exposure to the program. Yet this should not necessarily be interpreted as a failure of the 
program. In fact, as long as outcomes did not deteriorate after the first year, the lack of change 
could be considered a good outcome — because the comparison for these academies is not 
between no exposure and some exposure to the program but, rather, between some exposure and 
more exposure to it. As seen in the participation analysis above, participation between the first 
and second years did not increase as much as during the first year. Rather, for many of the 
indicators, participation rates were sustained into the second year at more or less the same rates 
that were achieved in the first year."^ 


^As shown in Appendix Table C.ll, there is no corresponding difference in the analysis of individual- 
level growth in this outcome, casting some doubt on the possibility that the cohort-level growth that is shown 
in Appendix Figure C.2 can be attributed to ECCO. 

"^Although these results appear to suggest that ECCO had the most effect on outcomes in the first two se- 
mesters and that the effect was sustained in the second year, the results should be treated with caution. To draw 
such a conclusion assumes that the first-year academies and their students are similar to the second-year 
academies and their students. While the analyses controls for some observable differences, the unobserved 
differences related to the outcomes make direct comparisons between the outcomes for the students in the two 
pairs of academies difficult. 
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The Career Aeademies Projeet 
Table 5.2 


Changes in Average Scores on Scales Measuring Short-Term Outcomes 
from Spring 2010 to Spring 2011, Second-Year Academies (COA and DSA) 


Outcome 

Spring 2010 

Spring 2011 

Estimated 

Change 

Effect 

Size 

P-Value 
for Change 

10th and 11th grades 






Semesters in ECCO program 

2 semesters 

4 semesters 




Engagement 

2.89 

2.87 

-0.01 

-0.03 

0.811 

Career awareness 

3.03 

2.95 

-0.08 

-0.10 

0.414 

College awareness 

2.97 

2.85 

-0.12 

-0.15 

0.192 

Self-assessed 21st-century skills 

3.33 

3.35 

0.02 

0.03 

0.742 

Total number of students’* 

155 

146 




12th grade 






Semesters in ECCO program 

0 semesters 

4 semesters 




Engagement 

3.01 

3.00 

-0.02 

-0.03 

0.870 

Career awareness 

2.85 

3.21 

0.36 

0.46 

0.016 ** 

College awareness 

3.19 

3.41 

0.22 

0.28 

0.138 

Self-assessed 21st-century skills 

3.34 

3.39 

0.05 

0.10 

0.506 

Total number of students’* 

66 

60 





SOURCE; MDRC calculations are from the ECCO spring student surveys. 

NOTES; The student outcomes presented in this table are calculated by averaging across a student's responses 
to relevant survey items. These survey items (and the resulting outcomes) are on a 4-point scale (1 = not at all 
true; 2 = not very true; 3 = sort of true; 4 = very true). This analysis is based on students who responded to 
both the spring 2010 and 2011 student surveys. The "Spring 2010" column presents the outcomes for these 
students in spring 2010. The "Spring 2011" column presents the outcomes for these students in spring 2011. 
The "Estimated change" column is the change in outcomes for these students from spring 2010 to spring 
2011. Hypothesis testing accounts for repeated observations of students across survey years. 

A two-tailed t-test is used for all statistical tests presented in this table. Effect sizes are based on the 
standard deviation of an outcome for all survey respondents in spring 2010 and spring 2011. Statistical 
significance levels are indicated as follows; *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

'“Due to missing values on some survey items, the number of students included in the analysis varies by 
outcome. The total number of students reported in the table are the number of students who responded to the 
survey. 

COA = Culinary Operations Academy; DSA = Digital Safari Academy. 
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The Career Aeademies Projeet 
Table 5.3 


Beliefs About Connections Between School and Future: 

Changes Over Time Across Consecutive Cohorts of Students 
(Tenth and Eleventh Grades), First-Year Academies (AOAT and HTM) 


Outcome (%) 

Spring 

2010 

Spring 

2011 

Estimated 

Change 

P-Value 
for Change 

Participation in ECCO program 

0 semesters 

2 semesters 



There is a connection between school and 
what 1 will do when I finish school 

51.6 

59.5 

7.9 

0.074 * 

What is learned in school is important for 
later in life 

80.1 

83.8 

3.7 

0.287 

Total number of students^* 

280 

274 




SOURCE; MDRC calculations are from the ECCO spring student surveys. 

NOTES; The student outcomes presented in this table are calculated by averaging across a student's 
responses to relevant survey items. These survey items (and the resulting outcomes) are on a 4-point scale (1 
= not at all true; 2 = not very true; 3 = sort of true; 4 = very true). This analysis is based on students who 
responded to both the spring 2010 and 2011 student surveys. The "Spring 2010" column presents the 
outcomes for these students in spring 2010. The "Spring 2011" column presents the outcomes for these 
students in spring 2011. The "Estimated change" column is the change in outcomes for these students from 
spring 2010 to spring 2011. Hypothesis testing accounts for repeated observations of students across survey 
years. 

A two-tailed t-test is used for all statistical tests presented in this table. Effect sizes are based on the 
standard deviation of an outcome for all survey respondents in spring 2010 and spring 2011. Statistical 
significance levels are indicated as follows; *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

“Due to missing values on some survey items, the number of students included in the analysis varies by 
outcome. The total number of students reported in the table are the number of students who responded to the 
survey. 

AOAT = Academy of Art and Technology; HTM = Center for Hospitality, Tourism, and Marketing. 


Interns’ and Employers’ Perceptions of Learning Outcomes from the 
Internship Program 

The summer internship program was the “capstone” work-based learning experience in 
the ECCO program. Students prepared for the internship in many ways, including participating 
in the half-day career exploration visits while in the tenth and eleventh grades prior to the 
internship. While in the six-week internship, students attended a weekly, half-day seminar with 
their peers. As illustrated by the qualitative information in Chapter 3, many students, their 
teachers, and their supervisors reported that the internship was a powerful experience that taught 
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them valuable lessons ranging from what the “real world is like” to how to interact effectively 
with colleagues and supervisors and to make more informed choices about the future. 

All interns — both those who completed an internship in 2010 and those who complet- 
ed one in 201 1 — were surveyed in the last week or two of the internship. Table 5.4 shows how 
they responded to a number of questions about the program and what they had learned. They 
were also asked whether they believed that their 21st-century skills had grown as a result of 
their internship experience. Within a week after the internship was over, supervisors were 
surveyed as well, and they were asked to rate the interns on the same set of skills. Table 5.5 
shows how student interns and their supervisors assessed the growth in 21st-century skills. 


The Career Academies Project 
Table 5.4 

Students Perceptions of Learning Outcomes from Internship Program, 


Pilot Academies 

Indicator Percentage 

Interng reports 

Had opportunities to practice skills that were learned in school the year before 77.4 

Learned or practiced skills that will help when returning to school 

in the fall 77.4 

Internship has influenced thinking about education or career goals 81 .9 

Helped decide to pursue a career in that field 55.3 

Helped realized that intern is not interested in pursuing a career in that field 18.8 

Helped understand the education and training needed to succeed in that field 68.2 

Would recommend the internshp program to a friend 100.0 

Total number of interns'* 90 


SOURCES: MDRC calculations are from the intern and employer surveys. 

NOTES: Responses are averaged across summer 2010 and summer 2011. The Academy of Art and 
Technology and the Center for Hospitality, Tourism, and Marketing started offering summer internships in 
Summer 201 1, so these academies have a smaller weight in the average. 

““Due to missing values on some survey items, the number of students included in the analysis varies by 
measure. The total number of interns reported in this table is the number of students who responded to the 
intern survey. 
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The Career Aeademies Projeet 
Table 5.5 


Percentages of Interns and Supervisors Who Reported Growth in Intern^ 
21st-Century Skills, Pilot Academies 


Skilf 

Percentage 

Interns Supervisors 

Solve problems and make decisions 

96.4 


28.3 

Think creatively 

97.6 


11.7 

Learn through research 

83.3 


6.7 

Ask appropriate questions 

94.0 


23.3 

Observe carefully 

96.4 


10.0 

Listen carefully 

100.0 


13.3 

Speak distinctly 

96.4 


21.7 

Write clearly 

71.4 


14.3 

Use visuals appropriately and skillfully 

95.2 


11.9 

Be precise and accurate 

94.0 


16.9 

Cooperate with others 

98.8 


15.0 

Persist when faced with a difficult task or situation 

95.2 


23.7 

Give and receive feedback 

96.4 


18.3 

Negotiate challenges and resolve conflicts 

92.9 


12.3 

Plan and prioritize to reach a goal 

95.2 


18.6 

Use new and emerging technology 

88.1 


3.6 

Use responsible behavior regarding technology 

94.0 


1.8 

Manage time effectively 

97.6 


11.7 

Demonstrate appropriate behavior and attire 

96.4 


8.3 

Recognize and respond to workplace challenges 

97.6 


8.8 

Show initiative and work independently 

100.0 


22.0 

Be reliable and dependable 

98.8 


4.9 

Understand the career field in which student interned 

97.6 


21.3 

Number of interns and supervisors'” 

90 
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Table 5.5 (continued) 

SOURCES; MDRC calculations are from the intern and employer surveys. 

NOTES; Responses are averaged across summer 2010 and summer 2011. The Academy of Art and 
Technology and the Center for Hospitality, Tourism, and Marketing started offering summer internships in 
summer 201 1, so these academies have a smaller weight in the average. 

'‘These measures are calculated based on a student's and supervisor's responses to items asking them to what 
extent the intern's skills improved over the course of the summer. Interns' responses are on a 4-point scale 
(none; a little; some; a lot). Interns are classified as having perceived growth in their skills if their response to 
an item is "a little," "some," or "a lot." Supervisors are classified as having perceived growth in interns' skills if 
their response to an item is "yes." 

’’Due to missing values on some survey items, the number of interns and supervisors included in the 
analysis vary by measure. The total number of interns reported in this table is the number of students who 
responded to the intern survey. The total number of supervisors reported in this table is the number of interns 
for whom a supervisor responded to the employer survey. In cases where an intern was rated by more than one 
supervisor, one rating was chosen at random. 


In general, the student interns reported learning much from the experience, reinforcing 
the conclusions in Chapters 3 and 4. While supervisors reported growth as well in many of the 
21st-century skills, their assessments are considerably less positive than the interns’ assessments 
of their own growth. This points to the value of using supervisors’ assessments of interns in 
addition to self-assessments as an indicator of the attainment of intended learning outcomes 
from internships. Highlights from Tables 5.4 and 5.5 include the following: 

• 82 percent of the interns reported that the internship experience had helped 
them think about their education and career goals, although fewer — 68 per- 
cent — said that it had helped them understand what training is needed to 
succeed in that field. 

• 77 percent of the interns reported that they had learned or practiced skills that 
would help them succeed in school. 

• 68 percent of the interns indicated that the internship had helped them under- 
stand the education and training needed to pursue a career in the same field 
as the internship. 

• Nearly all the interns reported at least some growth in the 21st-century skills 
listed in Table 5.5. In fact, the only skills for which less than 90 percent of 
the interns reported growth are how to “learn through research,’’ “write clear- 
ly,’’ and “use new and emerging technology.’’ 

• Supervisors were considerably less likely than the interns to indicate that in- 
terns had shown growth in 21st-century skills. Skills for which at least one in 
five interns had grown according to their supervisors include “solve prob- 
lems and make decisions,’’ “ask appropriate questions,’’ “speak distinctly,’’ 
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“persist when faced with a difficult task or situation,” “show initiative and 
work independently,” and “understand the career field in which the student 
interned.” The percentages of supervisors who reported intern growth in the 
remaining skills range from 2 percent to 19 percent. 


Interpreting the Findings 

The analysis of participation in the ECCO program and student outcomes that is presented 
above is exploratory only, due to the lack of a strong comparison group and small sample sizes 
for some of the analyses. This limits the conclusions that can be drawn. Nonetheless, the results 
of the analysis with respect to whether ECCO increased participation rates in college explora- 
tion activities and in career exploration activities point firmly to the plausibility that it was 
program participation that can account for these increases. Further, these results are consistent 
with the findings presented in Chapter 3 that the academies were able to put into place a set of 
college and career exploration activities that either did not exist earlier or did not touch very 
many students. This chapter presents reasonably strong evidence that students in the ECCO 
program participated in these activities at higher rates — and, for some activities, such as 
internships and visits to college campuses, at dramatically higher rates. The results also suggest 
that most growth in participation in these activities occurred during the first year of ECCO 
implementation, with growth tapering off in the second year. 

Evidence of the plausibility of the program’s effects on growth in career awareness and 
college awareness, engagement, 21st-century skills, and belief in the connection between school 
and future careers is mixed. Students showed growth in college and career awareness and were 
more likely to see the connection between school and their futures during the first year of 
ECCO. But there was no significant change in either engagement or 21st-century skills. With 
the exception of college awareness, little change in any of these outcomes (beyond what came 
about in the first year) seems to have occurred in the second year of program participation. This 
may mean that ECCO may have contributed to students’ experiencing these types of activities 
and learning from them a year or two earlier than they might have otherwise. Most would agree 
that raising college and career awareness early in high school rather than waiting until the senior 
year can be beneficial, because students can take steps earlier to prepare for their futures. For 
example, if students learn that a four-year degree is necessary to achieve their career goals, then 
they might be more motivated to do well in their academic classes in tenth grade rather than 
trying to catch up with college preparatory courses in twelfth grade. 

Again, given the limitations of the data, it is difficult to know whether any of the ob- 
served growth is due to ECCO or results from students’ simply maturing, preexisting differ- 
ences in individual students or cohorts, student participation in other programs, or other factors. 
While some results are consistent with the theory that ECCO can account for at least some of 
the growth, this cannot be confirmed without a more rigorous study. 
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Chapter 6 

Conclusions 


As noted in Chapter 1, career academies are an enduring and effective high school reform 
model that has widespread and growing appeal among educators. Combining a more personal- 
ized learning environment with a career theme, a sequence of career-technical courses, interdis- 
ciplinary curricula, and work-based learning, career academies are uniquely positioned to 
prepare students for both postsecondary education and productive careers. They can be found in 
many high schools in most states, and they attract a wide range of students. Some school 
districts believe so strongly in career academies that they have moved to making them the mle 
rather than the exception, by encouraging large high schools to offer academies to all their 
students. A few states are following this lead and are introducing initiatives to encourage the 
scaling up of academies and similar programs. 

While academies have the stmctural characteristics to facilitate preparation for college 
and career, they often lack the tools and resources to create the learning opportunities to make 
this happen. Arguably, the most difficult component of the academy model to implement fully 
is work-based learning. Whether the experience involves a one-day visit by students to a 
workplace, a visit from a local employer to talk with students about their profession, or the 
capstone six-week internship, academies have stmggled to launch and maintain such activities 
at scale. These difficulties should not be surprising. With little or no funding to cover the extra 
time that teachers need to design and mn these programs, with no training on how to design or 
operate them, and typically with no access to materials and curricula, it is little wonder that the 
work-based learning component is often neglected and that so few students, if any, receive 
internships. The recent downturns in the economy have not made the challenges easier, as 
employers may be reluctant to take on any more risks than they already face. 

Academies also have not traditionally offered their students college awareness or explo- 
ration activities, over and beyond what the host high school offers. Usually, students have to 
take the initiative to sign up for college visits. Some high school counselors even view academy 
students as “career-bound” rather than “college-bound” and do not include them in college 
presentations or campus visits. Without training and curricula, academy teachers are ill- 
equipped to provide more than ad hoc counseling and advice to students about their postsecond- 
ary options and how to select and apply to a college. 

The purpose of the project described in this report was to design and test an intervention 
— the Exploring College and Career Options (ECCO) program — that would systematically 
build the capacity of academies to create or restore career and college exploration activities for 
all their students each year that they are in the academy. The primary question that drove the 
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research is simply whether academies — if given the resources, materials, and support to do so 
— can launch and maintain a high-quality work-based learning program for all students. 

Nineteen academies with nonexistent or weak work-based learning programs in six 
school districts were recmited to take on the challenge of implementing ECCO. Of those, one 
academy dropped out before program implementation began, and two withdrew several months 
into the project. Of the remaining 16 ECCO academies, all were able to significantly increase or 
improve the quality of career and college exploration activities, compared with any such 
activities that they might have offered before ECCO. More students experienced more activi- 
ties. Most programs stabilized and improved over time. Many of the academies and the districts 
expressed interest in continuing the ECCO program, especially if funding were available to 
cover teachers’ additional time. 

That being said, the work of building career and college exploration programs was not 
without significant challenges, even when academies had access to generous resources, as was 
the case for the four pilot academies in this project (Chapter 3). Fitting new curricula into 
already-packed courses (whether it was a career-technical course or an academic course) was 
difficult. Scheduling off-campus visits to workplaces and college campuses confronted the 
academies with a set of logistical challenges. Finding suitable work sites where employers could 
offer stmctured learning opportunities to a group of young people took time, effort, and skill. 
The implementation of ECCO was not perfect. But the progress and accomplishments of the 
academies in implementing the program leave no doubt that it succeeded in increasing the 
capacity of academies to build career and college exploration programs that could reach all 
students, regardless of their academic records, whether they had ever thought about college, and 
their level of maturity. Rates of participation in career and college exploration activities and the 
learning that occurred as a result increased significantly after ECCO was implemented. This 
success is especially remarkable in the context of a fiscally challenging time for schools, 
districts, and employers. 

While the study reported here clearly establishes the feasibility of building the capacity 
of academies to offer career and college exploration activities, the evidence is less clear about 
whether increased participation in those new programs made a difference in the short-term 
outcomes that were predicted by the theory of change (Chapter 1, Figure 1.1). In interviews and 
focus groups, teachers and students overwhelmingly reported positive experiences and cited 
many examples of the growth and learning that took place. The analysis of scores on scales 
measuring changes in career and college awareness points to some statistically significant and 
substantial differences for some indicators, especially in the first year of participation in ECCO. 
But the magnitude of these differences is modest, and no meaningful improvement occurred for 
two of the four outcomes: engagement in school and growth in 21st-century skills. Importantly, 
without a more rigorous design, it is difficult to know whether or not ECCO made a difference 
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in the short-term outcomes predicted by the theory of change. This finding, in combination with 
the result that the academies succeeded in implementing this program, leads to a number of 
implications for educators and researchers. 

First, while it is feasible for academies to launch a cohesive program of college and ca- 
reer exploration curricula and activities, this is unlikely to happen without support. Teachers 
need resources, time, and training to make it happen. This is particularly tme for developing 
work-based learning programs that directly involve employers. 

Second, with resources and support, academies in this project were generally able to place 
about half of their juniors in internships. The percentage of juniors who were placed varied, but 
the number rarely exceeded 20. Although a handful of students were not placed because of a 
shortage of employer hosts, most of the juniors who opted out of internships did so because of 
competing demands on their time. Summer school, transportation costs, the need to earn money, 
and family obligations were the most common reasons for opting out. This implies that alterna- 
tives to the traditional internship model such as was used in ECCO — whereby one student is 
placed with one employer dirring the summer — should be considered. Many academies are 
already experimenting with such alternatives as “virtual internships,” whereby students interact 
with employers either online or in person but without leaving the school campus; group intern- 
ships, which place four or five students at a work site and they work as a team; employer mentor- 
ing programs that match a working adult with a single student; and school-based enterprises, such 
as a student-mn business that gives students a work-based learning experience without leaving 
campus. Ideally, the traditional internship should be just one of several outcome-driven options 
that academies can offer to provide students with work-based learning experiences. 

Third, college and career exploration activities should be tightly integrated with the to- 
tality of the academy experience. These activities are best not relegated to the career-technical 
teacher alone. English, history, and math teachers should be involved with teaching an in-class 
career and college exploration curriculum, and they should find ways to integrate this material 
with their subject curriculum, accompany students on visits to the workplace and to college 
campuses, and create and monitor internships. When this occurs, students are likely to better 
understand the connections between what they are learning in their math and English classes 
and their futures, leading to deeper engagement in school and better postsecondary outcomes. 

Fourth, a program like ECCO could be adapted to build the capacity of other education- 
al programs to mount, scale, and sustain career and college exploration activities and to assess 
their effects on key student outcomes. Preparing young people for success in the workplace and 
college is part of the mission of many programs serving young people, including regular high 
schools, extended learning programs, youth development programs, job training programs, 
community colleges, middle schools, after-school programs, and area vocational schools. 
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Learning what it takes to move beyond the rhetoric and make career and college readiness 
programs happen for low-income young people in a variety of settings would be an invaluable 
contribution to the field. 

Fifth and finally, many questions remain about the effects that career and college explo- 
ration and preparation programs in high school have on long-term outcomes, such as enrollment 
in and completion of postsecondary education and participating in the labor market. To date, no 
rigorous impact studies have been conducted to test the impact of internships and other work- 
based learning experiences on these long-term outcomes. While such programs hold promise to 
make a difference in the lives of at-risk, low-income students, without more rigorous evidence 
than is presented in this report, their tme potential remains unknown. 

For more information about the ECCO program and how to access it, see 
www.connectedstudios.org/ecco. 
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Appendix A 

Data Sources and Survey Response Analysis 




The first two sections of Appendix A describe the quantitative and the qualitative data sources 
used in this report on the Exploring Career and College Options (ECCO) program, including 
their purpose, timing, target population, and response rates, when applicable. The third section 
of this appendix presents a response analysis for the student survey; this is detailed, since it is 
the main source of information for evaluating ECCO’s potential for improving student partici- 
pation and outcomes. The analysis includes survey response rates by year and by grade level, a 
comparison of the characteristics of respondents in each survey year, a comparison of survey 
respondents and nonrespondents, and the characteristics of survey respondents in each of the 
four pilot academies discussed in Chapter 3. 


Quantitative Data Sources 

Appendix Table A.l summarizes the following quantitative data sources used in this report: 

• Class rosters. Class rosters were collected to identify students who were en- 
rolled in the pilot academies in spring 2010, fall 2010, spring 2011, and fall 
2011. These rosters were also used to identify attrition patterns over time, to 
assess response rates for the spring student survey (discussed in the next sec- 
tion), and as a data source for students’ background characteristics (including 
race/ethnicity, gender, and grade). ECCO coordinators provided the rosters at 
the end of each semester, along with a list of students who had left the acad- 
emy between semesters and the reasons why they had left. 

• Student surveys. ECCO was expected to affect outcomes that are typically 
not collected in school records, such as college and career awareness 
measures. Therefore, a survey was used to collect information on these out- 
comes as well as students’ background characteristics. Specifically, the sur- 
veys collected information on parental education, prior academic achieve- 
ment, participation in and level of college and career awareness and 
exploration, career or college exploration and preparation activities, curricu- 
lum, engagement, opportunities to practice 21st-century skills, and growth in 
those skills. All tenth- through twelfth-grade students who were enrolled in 
the pilot academies were surveyed in their classrooms in late April 2010 and 
2011. Across all grades (10 through 12), 90 percent of students who were en- 
rolled in the pilot academies responded to the survey, though the response 
rates differ across survey years. (See the next section of this appendix.) 

• Intern surveys and intern employer surveys. Offering all juniors an oppor- 
tunity to participate in an internship in the summer before their senior year or 
in their senior year is a core component of the ECCO model. Therefore, a 
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The Career Academies Project 
Appendix Table A.l 


Quantitative Data Sources for Student Characteristics and Outcomes, Pilot Academies 


Data Source 

Measure/Domain 

Timing 

Target Population 

Academy class rosters 

Race/ethnicity, gender, and grade 

Spring 2010 and 2011 

Students in grades 10-12 

Student surveys 

Demographic information (mother's education, 
grade retention, course marks in previous grades) 
Participation in career and college awareness activities 
Student outcomes (engagement, college and career 
awareness, self-assessment of skills) 

Spring 2010 and 2011 

Students in grades 10-12 

Intern participation logs 

Internship applicants, intern placement, and 
and completion status 

Spring/summer 2010 
and 2011 

1 Ith-graders (grade 1 1 in 
2010 or 2011) 

Intern surveys 

Implementation of internship components 
Perceptions of internship quality and usefulness 
Self-assessment of growth in skills 
Self-assessment of growth in career/college awareness 

Summer 2010 and 2011 

Students placed in 
internships 

Intern employer surveys 

Implementation of internship components 
Perceptions of internship quality and usefulness 
Assessment of intern's growth in skills 

Summer 2010 and 2011 

Interns' supervisors 

Student withdrawal forms 

Reasons for leaving academy 

Summer/ fall 2010 
and 2011 

Students who left academy 
after spring 2010 or 201 1 



survey was administered to student interns and their employers in summer 2010 
and in summer 2011 in the last week of their six-week internships. The survey 
included questions on the interns’ experiences in the internship program, assess- 
ment of their growth in 21st-century skills, and knowledge of career and college 
options. The intern employer survey was administered to the employers who 
hosted these interns. Part I of the instalment asked employers to respond to ques- 
tions about the program itself (for example, did they feel well prepared and sup- 
ported, would they do it again, and so on), as a means of measuring program im- 
plementation. Part II of the form asked employers to rate the performance of each 
of their interns, particularly with respect to the strength of 21st-century skills. Of 
the 106 internship placements in summer 2010 and summer 2011, 90 students 
(85 percent) completed the intern survey, and employer ratings were received 
from about 53 supervisors (50 percent). Of the 106 students who participated in 
an internship, the skills of 65 interns (61 percent) were evaluated by their em- 
ployers. 


Qualitative Data Sources 

The following qualitative data sources were used to describe the implementation experiences of 
the pilot academies (Chapter 3) and the scale-up academies (Chapter 4): 

• Observations of summer planning visits. Summer planning visits served as 
the initial training sessions for district employees and ECCO coordinators. 
Observations of these planning sessions were conducted several times over 
the course of the project. 

• Telephone interviews. Stmctured phone interviews with district facilitators 
and ECCO coordinators were conducted most semesters. 

• Site visits. In-person site visits by the research team involved focus groups 
and interviews with school district and academy staff 

• Implementation status reports. During semesters when no site visits were 
planned, each academy was asked to complete an implementation status re- 
port to record which lessons and activities were implemented. 

• Program developer reports. Monthly reports from the program developers 
summarized the technical assistance that was provided to the sites and their 
progress with implementation. 
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Interviews with program developers. Interviews were conducted once a 
year with the program developers to obtain their perspectives and insights 
about ECCO implementation. 


Response Analysis for the Student Survey 

Chapter 5 evaluates ECCO’s potential for improving student outcomes by looking at the 
changes over time in measures of student participation and outcomes that are derived from the 
student survey (that is, differences in outcomes between students who were surveyed in spring 
2010 and those who were surveyed in spring 2011). Given this analytical strategy, it is im- 
portant to examine whether response rates and respondents’ characteristics are similar across the 
two survey years; if not, then differences in student outcomes across survey years could simply 
be due to differences in respondents’ characteristics, rather than representing real differences in 
outcomes. This could compromise the “internal validity’’ of the findings reported in Chapter 5 
as a source of information for evaluating ECCO’s potential for improving student outcomes. 

Appendix Table A.2 presents response rates for the student survey in the pilot acade- 
mies, by grade level, for each year of the survey. These response rates are calculated by compar- 
ing the number of students who responded to the survey and the number of students in the class 
rosters. As seen in this table, the response rate is generally high, above 86 percent. However, 
there is a statistically significant difference in response rates across survey years: 91 percent in 
spring 2010, compared with 88 percent in spring 2011. This is due primarily to a between-year 
difference in response rates among twelfth-grade students. 

This difference in the response rates raises the possibility that the characteristics of sur- 
vey respondents also differ across survey years. Accordingly, Appendix Table A.3 compares 
selected characteristics of survey respondents in spring 2010 with those of respondents in spring 
201 1. As seen in this table, students do differ statistically with respect to several characteristics, 
such as race/ethnicity, though the magnitude of these differences in not large; for example, 87 
percent of respondents in spring 2010 are from a racial/ethnic minority group, compared with 
91 percent of respondents in spring 2011. 

Two analytical strategies were used to correct for these differences in respondents’ 
characteristics. First, a regression analysis was used to adjust the estimated changes in student 
outcomes for between-year differences in characteristics. (Appendix C presents the statistical 
model for cohort-level changes over time.) Of course, these regression adjustments may not 
completely resolve the problem: If respondents differ with respect to their observed characteris- 
tics, then they probably also differ with respect to their unobserved characteristics. Therefore, it 
remains possible that estimates of changes over time in student outcomes across consecutive 
cohorts of students, as presented in Chapter 5, could be due to differences in unobservable 



The Career Aeademies Projeet 
Appendix Table A.2 


Student Survey Reponse Rates, by Year and Grade Level, Pilot Academies 


Grade Level 

Spring 

2010 

Spring 

2011 

Estimated 

Difference 

P-Value 
for Change 

All grades (10-12) 





Response rate (%) 

91.4 

88.3 

-3.1 

0.060 * 

Number of students enrolled 

695 

666 



Grade 10 





Response rate (%) 

88.1 

87.9 

-0.2 

0.949 

Number of students enrolled 

277 

248 



Grade 1 1 





Response rate (%) 

91.8 

90.2 

-1.6 

0.551 

Number of students enrolled 

208 

224 



Grade 12 





Response rate (%) 

95.2 

86.6 

-8.6 

0.002 *** 

Number of students enrolled 

210 

194 




SOURCES; MDRC calculatations are from the ECCO spring student surveys and academy class rosters. 

NOTES; Values in the "Spring 2010" and "Spring 2011" columns show the percentage of students enrolled in 
the academies (based on spring rosters) who answered the student survey in the spring of each survey year. 
The "Estimated Difference" column is the difference in response rates between survey years. 

A two-tailed t-test was used to test whether this difference is statistically significant. Statistical 
significance levels are indicated as follows; *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Hypothesis tests account for the fact some students are observed in both survey years. 


characteristics of respondents rather than a real improvement in student outcomes. To resolve 
this problem, the plausibility of ECCO’ s effects was also evaluated using a second analytical 
approach, which looks at whether there are changes over time in outcomes for individual 
students who responded to both rounds of the survey. (See Appendix C for the statistical model 
for student-level changes over time.) Because the same students are compared in both time 
periods, estimates of individual-level changes over time are not confounded with differences in 
either observed or unobserved respondent characteristics. As discussed in Appendix C, the 
findings from this additional analysis generally corroborate the findings presented in Chapter 5. 

Another important analytical issue is whether the survey findings are “externally valid,” 
that is, whether the findings can be generalized to nonrespondents and, more broadly, to the 
target population of all enrolled students. Accordingly, Appendix Table A.4 compares selected 
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The Career Academies Project 
Appendix Table A.3 


Seleeted Charaeteristies of Survey Respondents, by Survey Year, 

Pilot Aeademies 


Characteristic (%) 


Spring 

2011 

Estimated 

Difference 

P- Value 
for Change 

Racial/ethnic minority” 

87.4 

91.2 

3.8 

0.031 ** 

Male 

47.3 

45.4 

-1.9 

0.512 

Mother's educational attainment 




0.001 *** 

Less than high school 

43.8 

44.9 

1.1 


High school diploma or GED certificate 

31.1 

32.4 

1.3 


Postsecondary credential 

24.9 

22.2 

-2.6 


Ever held back a grade 

22.6 

23.8 

1.2 

0.624 

Received a D or below in English or math 





in the previous school year 

33.0 

34.5 

1.5 

0.624 

Grade level 




0.253 

Grade 10 

38.4 

37.1 

-1.4 


Grade 1 1 

30.1 

34.4 

4.3 


Grade 12 

31.5 

28.6 

-2.9 


Academy 




0.874 

Academy of Art and Technology 

38.9 

37.8 

-1.1 


Center for Hospitality, Tourism, and Marketing 

26.3 

27.2 

0.9 


Culinary Operations Academy 

10.7 

9.7 

-1.0 


Digital Safari Academy 

24.1 

25.3 

1.2 


Omnibus test (chi-squared = 




0.021 ** 

Total number of students” 

635 

588 




SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: Values in the "Spring 2010" column are the characteristics of students who responded to the spring 
2010 student survey. Values in the "Spring 2011" column are the characteristics of students who responded to 
the spring 2011 survey. The "Estimated difference" column is the difference in characteristics between the 
two groups of students. 

A two-tailed t-test was used to test whether these differences are statistically significant. Statistical 
significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Hypothesis tests account for the fact the some students are observed in both survey years. 

“This includes students who are black, Hispanic, Asian, or "Other" race./ethnicity. 

'“A chi-squared test was used to determine whether there is a systematic difference between the two groups 
of students, across all characteristics included in this table as well as indicators of whether data were missing 
on each of these characteristics. 

"Due to missing values on some survey items and characteristics from the rosters, the number of students 
included in the analysis varies by characteristic. The total number of students reported in the table are the 
number of students who responded to the survey. 
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The Career Academies Project 
Appendix Table A,4 


Selected Characteristics of Survey Respondents and Nonrespondents, 

Pilot Academies 



Survey 

Non- 

Estimated 

P-Value 

Characteristic (%) 

Respondents 

Respondents Difference for Change 

Soring 2010 





Male 

47.3 

58.3 

-11.0 

0.103 

Racial/ethnic minority’* 

87.4 

86.7 

0.7 

0.874 

Grade level 




0.024 ** 

Grade 10 

38.4 

55.0 

-16.6 


Grade 11 

30.1 

28.3 

1.7 


Grade 12 

31.5 

16.7 

14.8 


Academy 




<0.001 *** 

Academy of Art and Technology 

38.9 

25.0 

13.9 


Center for Hospitality, Tourism, and Marketing 

26.3 

56.7 

-30.4 


Culinary Operations Academy 

10.7 

6.7 

4.0 


Digital Safari Academy 

24.1 

11.7 

12.4 


Omnibus test (chi-squared = 33.4'’ 




<0.001 *** 

Total number of students” 

635 

60 



Soring 2011 





Male 

45.4 

45.3 

0.1 

0.986 

Racial/ethnic minority” 

91.2 

94.7 

-3.4 

0.312 

Grade level 




0.512 

Grade 10 

37.1 

38.5 

-1.4 


Grade 11 

34.4 

28.2 

6.1 


Grade 12 

28.6 

33.3 

-4.8 


Academy 




<0.001 *** 

Academy of Art and Technology 

37.8 

24.4 

13.4 


Center for Hospitality, Tourism, and Marketing 

27.2 

64.1 

-36.9 


Culinary Operations Academy 

9.7 

2.6 

7.1 


Digital Safari Academy 

25.3 

9.0 

16.4 


Omnibus test (chi-squared = 39.5 




<0.001 *** 

Total number of students” 

588 

78 




(continued) 
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Appendix Table A.4 (continued) 


SOURCES; MDRC Calculations are from the ECCO spring student surveys and academy class rosters. 

NOTES; Values in the "Survey Respondents" column are the characteristics of students who responded to the 
student survey in a given survey year. Values in the "Nonrespondent" column are the characteristics of 
students who are enrolled in the academy (based on the spring rosters) but who did not respond to the student 
survey. The "Estimated difference" column is the difference in characteristics between the two groups of 
students. 

A two-tailed t-test was used to test whether these differences are statistically significant. Statistical 
significance levels are indicated as follows; *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

'‘This includes students who are black, Hispanic, Asian, or "Other" race/ethnicity. 

’’A chi-squared test was used to determine whether there is a systematic difference between the two groups 
of students, across all characteristics included in this table as well as indicators of whether data were missing 
on each of these characteristics. 

“Due to missing values in the demographic characteristics reported in the rosters, the number of students 
included in the analysis varies by characteristic. The total numbers of students reported here are for the full 
sample of students enrolled in the academies or who responded to the survey. 


characteristics of respondents and nonrespondents (that is, the students who were enrolled in the 
academies but who did not respond to the surveys). As seen in the table, respondents and 
nonrespondents are similar with respect to demographic characteristics (gender and 
race/ethnicity). The two groups differ, however, with respect to grade level (respondents were 
more likely than nonrespondents to be in twelfth grade and, therefore, were older) and with 
respect to academy (respondents were more likely than nonrespondents to be enrolled in certain 
academies rather than others). 

Strictly speaking, this means, on the one hand, that the findings presented in this report 
— which are based on survey respondents — may not be generalizable to nonrespondents. On 
the other hand, because the percentage of nonrespondents is small, the distribution of respond- 
ents across grade levels and academies is, in fact, similar to that of all enrolled students (the 
target population); see Chapter 1, Table 1.2. Therefore, the findings in Chapter 5 are likely to be 
adequately generalizable to all the students enrolled at the academies. 

Appendix Table A.5 presents selected characteristics of students served by each acade- 
my, based on the sample of survey respondents. As seen in this table, the characteristics of 
respondents varied substantially across academies. For example, the percentage of students 
from a racial/ethnic minority group ranges from 52 percent to 100 percent, while the percentage 
of students ever held back ranges from 7 percent to 33 percent. 
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The Career Academies Project 
Appendix Table A.5 


Selected Characteristics of Survey Respondents, by Academy, Pilot Academies 


Characteristic (%) 

Academy of 
Art and 
Technology 

Culinary 

Operations 

Academy 

Center for 

Digital Safari Hospitality, Tourism, 
Academy and Marketing 

Racial/ethnic minority^' 

90.6 

52.0 

90.7 

100.0 

Male 

49.9 

60.2 

46.2 

36.4 

Mother's educational attainment 

Less than high school 

61.3 

26.2 

36.4 

35.0 

High school diploma or GED certificate 

24.3 

37.4 

30.0 

42.1 

Postsecondary credential 

13.8 

36.4 

33.6 

22.5 

Ever held back a grade 

26.4 

27.0 

6.7 

32.7 

Received a D or below in English 

or math in the previous school year 

45.1 

28.9 

33.8 

18.0 

Grade level 

Grade 10 

38.0 

40.8 

34.4 

39.4 

Grade 1 1 

30.9 

32.0 

35.1 

31.2 

Grade 12 

31.1 

27.2 

30.5 

29.4 

Total number of students'’ 

469 

125 

302 

327 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: Values in the table are the characteristics of students who responded to the student survey in spring 2010 
or spring 2011. 

“This includes students who are black, Elispanic, Asian, or "Other" race/ethnicity. 

’’Due to missing values on some survey items and characteristics from the rosters, the number of students 
included in the analysis varies by characteristic. The total numbers of students reported here are the number of 
students who responded to the survey. 
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Appendix B 

Additional Findings About Implementation 




The Career Aeademies Projeet 
Appendix Table B.l 


Number of Students Wbo Participated in Summer Internships, 
by Academy, 2010 and 2011 Combined, Pilot Academies 



1 Ith-Graders 

Applied to 
Internship 

Placed in 
Internship 

Completed 

Internship 

Summer 2010 

Culinary Operations Academy 

28 

15 

14 

13 

Digital Safari Academy 

56 

28 

26 

20 

Total 

84 

43 

40 

33 

Summer 2011 

Academy of Art and Technology 

81 

30 

25 

23 

Center for Hospitality, Tourism, and Marketing 

74 

22 

9 

7 

Culinary Operations Academy 

13 

6 

6 

5 

Digital Safari Academy 

56 

27 

26 

21 

Total 

224 

85 

66 

56 

Both summers 

Academy of Art and Technology 

81 

30 

25 

23 

Center for Hospitality, Tourism, and Marketing 

74 

22 

9 

7 

Culinary Operations Academy 

41 

21 

20 

18 

Digital Safari Academy 

112 

55 

52 

41 

Total 

308 

128 

106 

89 


SOURCES; The MDRC calculation of the number of juniors is from academy rosters. Calculations of student 
participation, placement, and completion are from ECCO coordinators’ annual reports on the internship 
program. 
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The Career Academies Project 
Appendix Table B,2 

Time Spent by ECCO Coordinators, Pilot Academies 


Task 

Hours per Week 

ECCO coordinators time, bv task 

Participating in training or coaching 

2.0 

Planning to teach workshops 

1.4 

Planning for career exploration visits 

1.4 

Planning for college visits 

1.0 

Securing internship sites and projects 

2.5 

Visiting/monitoring internships 

2.9 

Planning internship seminars 

1.3 

Average across all tasks 

1.8 

ECCO coordinators time, bv semester 

Fall 

10.5 

Spring 

14.1 

Summer 

13.2 

Average across all semesters 

12.6 


SOURCE: MDRC calculations are from ECCO coordinators' time-use logs completed weekly 
for one year, from fall 2010 through fall 2011. 
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Appendix C 

Additional Analyses of Student Outcomes and 
Methodological Explanations 


Appendix C provides additional findings about ECCO’s potential to improve student outcomes 
as well as some methodological explanations. The first section presents additional exhibits 
associated with the primary analyses in Chapter 5. The second section looks at the changes over 
time in student outcomes for individual students who responded to both rounds of the survey, as 
a way of bringing further evidence to bear on ECCO’s potential for improving student out- 
comes. The third section of the appendix describes the statistical models used for all analyses 
presented in this report. 




Supplementary Exhibits for Chapter 5 


The Career Academies Project 
Appendix Table C,1 

Short-Term Student Outcomes: Constructs and Survey Items 


Construct and Definition^ 

Survey Item 

Career awareness 

Knowledge of career options, understanding of 
levels of educational attainment and skills needed 
for certain careers, and an understanding of steps 
needed to achieve career goals 

-I have a good idea of what kind of career I want to have 
-I have a good idea of what level of education is required for 
the kind of career I want 

-I have a good idea of what experiences and skills are required 
for the kind of career I want 

-I have a plan for what I’m going to do get the career I want 

College awareness 

Awareness of postsecondary options and 
knowledge of how to apply for and pay for college 

-I have a good idea of what I need to do for college 
-I have a good idea of what I need to do to pay for college 
-I know which college I want to go to 

21st-centnry skills 

Self-assessed strength in noncognitive, transferable 
skills 

I am good at: 

-collecting, organizing, and interpreting information to answer 
a question 

-solving problems and making decisions 

-thinking creatively 

-cooperating with others 

-communicating information clearly 

-planning and prioritizing to reach a goal 

-working independently 

-managing my time and staying organized 

-reflecting upon and evaluating my own work 

Engagement in school 

Level of in interest in and effort devoted to 
learning at school 

-I feel bored during most of my classes (reversed)*” 

-I pay attention in class 

-I work very hard on my schoolwork 

In school, we learn about things that I want to learn a lot more 
about 

Awareness of connection between school and 
fntnre 

Belief in the connection between knowledge, 
skills, and experiences gained in high school with 
preparing for college and career^ 

-There is a strong connection between what I am studying in 
school and what I want to do when I finish high school 
-I think the things I’m learning in school are going to be very 
important for my later life 


NOTES: “Scores for career awareness, college awareness, 21st-century skills, and engagement in school are based 
on calculations of average responses to a set of survey items on a 4-point scale, ranging from l(not true at all) to 4 
(very true). 

'^In order to make this item consistent with the other survey items, this item was reverse coded. Reverse coding is 
used when a negative item is grouped with positive items. Therefore, the item “I feel bored during most of my 
classes” was coded so that a 1 rating meant “very true” and a 4 rating meant “not true at all.” 

“This is calculated as the percentage of students who responded “sort of true” or “very true.” 
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The Career Academies Project 
Appendix Table C,2 

Beliefs About Connections Between School and Future: 
Changes Over Time Across Consecutive Cohorts of Students, 
Second-Year Academies (COA and DSA) 


Outcome (%) 

Spring 

2010 

Spring 

2011 

Estimated 

Change 

P- Value 
for Change 

10th and 11th grades 





Participation in ECCO program 

2 semesters 

4 semesters 



There is a connection between school and 
what I will do when I finish school 

57.2 

55.6 

-1.7 

0.781 

What is learned in school is important for later 
in life 

79.2 

77.3 

-1.9 

0.692 

Total number of students'* 

155 

146 



12th grade 

Participation in ECCO program 

0 semesters 

4 semesters 



There is a connection between school and 
what I will do when I finish school 

62.1 

67.5 

5.4 

0.540 

What is learned in school is important for later 
in life 

81.8 

85.1 

3.3 

0.612 

Total number of students'* 

66 

60 




SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on survey respondents enrolled in a given grade in spring 2010 and spring 
2011. The outcomes presented in this table are based on whether a student responded that the statement is 
either "sort of true" or "very true" (as opposed to "not at all true" or "not very true"). The "Spring 2010" 
column presents the regression-adjusted outcomes of students who responded to the spring 2010 student 
survey. The "Spring 2011" column presents the regression-adjusted outcomes of students who responded 
to the spring 2011 survey. The "Estimated change" column is the difference in outcomes between the two 
groups of students. This estimated difference is adjusted for the following student characteristics: 
academy fixed effects, whether a student is a racial/ ethnic minority, gender, mother's educational 
attainment, whether a student was ever held back, whether the student received a D or below in English 
or math in the previous school year, and missing indicators for each of these characteristics. 

A two-tailed t-test is used for all statistical tests presented in this table. Statistical significance levels 
are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

''Due to missing values on some survey items, the number of students included in the analysis varies 
by outcome. The total number of students reported in the table are the number of students who responded 
to the survey. 

COA = Culinary Operations Academy; DSA = Digital Safari Academy. 
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The Career Academies Project 
Appendix Figure C.l 

Participation in Career Exploration Activities During the School Year: 

Comparison of Tenth- and Eleventh-Grade Cohorts Combined, with Two and Four Semesters of ECCO, 

Second- Year Academies (COA and DSA) 


□ Spring 2010 (N = 155) ■ Spring 2010 (N = 146) 

0% 20% 40% 60% 80% 


Attended guest speaker presentation about jobs or workplace skills 
Learned in class about how to write a good resume 
Did a practice job interview 

Learned in class about how to act appropriately in the workplace 

Learned in class about how to respond to workplace challenges like 
discrimination/harassment 

Had opportunities to practice or learn 21st-century skills 
Discussed with a teacher or counselor what career would be best for you 
Worked with a teacher or counselor to create a plan for your future 
Learned what level of education is needed for various careers 
Learned what level of education is needed for a career that interests you 
Visited a work site as part of a school activity 


I 74' 

61% * 


68 % 



1 65% 
r 64% 


65% 

■ 70% 


100 % 
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(continued) 


Appendix Figure C.l (continued) 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on survey respondents enrolled in a given grade in spring 2010 and spring 201 1. The white bars are the regression-adjusted 
outcomes of students who responded to the spring 2010 student survey. The dark bars are the regression-adjusted outcomes of students who responded to the 
spring 2011 survey. 

Outcomes are adjusted for the following student characteristics: academy fixed effects, grade fixed effects, whether a student belonged to a racial or ethnic 
minority, gender, mother's educational attainment, whether a student was ever held back, whether the student received a D or below in English or math in the 
previous school year, and missing indicators for each of these characteristics. 

A two-tailed t-test is used for all statistical tests presented in this figure. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 
percent; * = 10 percent. 

Due to missing values on some survey items, the number of students included in the analysis varies by activity. The numbers of students reported in the 
figure are the number of students who responded to the survey. 

COA = Culinary Operations Academy; DSA = Digital Safari Academy. 
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The Career Aeademies Projeet 
Appendix Figure C.2 

Partieipation in Career Exploration Aetivities During the Sehool Year: 
Comparison of Twelfth-Grade Cohorts with Zero and Four Semesters of ECCO, 
Seeond-Year Aeademies (COA and DSA) 


□ Spring 2010 (N = 66) ■ Spring 2010 (N = 60) 


0% 20% 40% 60% 80% 100% 


Attended guest speaker presentation about jobs or workplace skills 
Learned in class about how to write a good resume 
Did a practice job interview 

Learned in class about how to act appropriately in the workplace 

Learned in class about how to respond to workplace challenges like 
discrimination/harassment 

Had opportunities to practice or learn 2 1 st-century skills 
Discussed with a teacher or counselor what career would he best for you 
Worked with a teacher or counselor to create a plan for your future 
Learned what level of education is needed for various careers 
Learned what level of education is needed for a career that interests you 
Visited a work site as part of a school activity 
Had internship during this last school year or last summer 



(continued) 


Appendix Figure C.2 (continued) 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on twelfth-grade survey respondents enrolled in the pilot academies in spring 2010 and spring 2011. Each spring, students 
were asked to reflect on their activities during the past school year. The white bars are the regression-adjusted outcomes of students who responded to the 
spring 2010 student survey. The dark bars are the regression-adjusted outcomes of students who responded to the spring 201 1 survey. 

Outcomes are adjusted for the following student characteristics: academy fixed effects, grade fixed effects, whether a student belonged to a racial or 
ethnic minority, gender, mother's educational attainment, whether a student was ever held baek, whether the student reeeived a D or below in English or 
math in the previous school year, and missing indicators for each of these characteristies. 

A two-tailed t-test is used for all statistical tests presented in this figure. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 
percent; * = 10 percent. 

Due to missing values on some survey items, the number of students included in the analysis varies by activity. The numbers of students reported in the 
figure are the number of students who responded to the survey. 

COA = Culinary Operations Academy; DSA = Digital Safari Academy. 
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The Career Academies Project 
Appendix Figure C.3 

Partieipation in College Exploration Aetivities During the Sehool Year: 

Comparison of Tenth- and Eleventh-Grade Cohorts Combined, with Two and Four Semesters of ECCO, 

Seeond-Year Aeademies (COA and DSA) 


□ Spring 2010 (N= 155) ■ Spring 2010 (N = 146) 

0% 20% 40% 60% 80% 


Talked with a teacher or other adult at school about college 
Talked with a parent or other family member about college 



Attended a college fair 
Attended a presentation by college representatives 
Looked up information about colleges on the Internet 
Visited a 4-year college campus 
Visited a community college or career-technical school 
Talked with an adult about how to pay for college 
Took the P-SAT (or signed up to take it this summer) 



100% 


** 

92% 


(continued) 


Appendix Figure C.3 (continued) 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on survey respondents enrolled in a given grade in spring 2010 and spring 201 1. The white bars are the regression-adjusted 
outcomes of students who responded to the spring 2010 student survey. The dark bars are the regression-adjusted outcomes of students who responded to the 
spring 2011 survey. 

Outcomes are adjusted for the following student characteristics: academy fixed effects, grade fixed effects, whether a student belonged to a racial or ethnic 
minority, gender, mother's educational attainment, whether a student was ever held back, whether the student received a D or below in English or math in the 
previous school year, and missing indicators for each of these characteristics. 

A two-tailed t-test is used for all statistical tests presented in this figure. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 
percent; * = 10 percent. 

Due to missing values on some survey items, the number of students included in the analysis varies by activity. The numbers of students reported in the 
figure are the number of students who responded to the survey. 

COA = Culinary Operations Academy; DSA = Digital Safari Academy. 
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The Career Academies Project 
Appendix Figure C.4 

Partieipation in College Exploration Aetivities During the Sehool Year: 
Comparison of Twelfth-Grade Cohorts with Zero and Four Semesters of ECCO, 
Seeond-Year Aeademies (COA and DSA) 


n Spring 20 1 0 (N = 66) ■ Spring 20 1 0 (N = 60) 


Talked with a teacher or other adult at school about college 
Talked with a parent or other family member about college 

Attended a college fair 
Attended a presentation by college representatives 
Looked up information about colleges on the Internet 
Visited a 4-year college campus 
Visited a community college or career-technical school 
Talked with an adult about how to pay for college 
Took the P-SAT (or signed up to take it this summer) 
Took the SAT or ACT (or signed up to take it this summer) 
Applied to a community college or career-technical program 

Applied to a 4-year college 
Did anything else to prepare for college 
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Appendix Figure C.4 (continued) 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on twelfth-grade survey respondents enrolled in a given grade in spring 2010 and spring 2011. Each spring, students were 
asked to reflect on their activities during the past school year. The white bars are the regression-adjusted outcomes of students who responded to the spring 
2010 student survey. The dark bars are the regression-adjusted outcomes of students who responded to the spring 201 1 survey. 

Outcomes are adjusted for the following student characteristics: academy fixed effects, grade fixed effects, whether a student belonged to a racial or ethnic 
minority, gender, mother's educational attainment, whether a student was ever held back, whether the student received a D or below in English or math in the 
previous school year, and missing indicators for each of these characteristics. 

A two-tailed t-test is used for all statistical tests presented in this figure. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 
percent; * = 10 percent. 

Due to missing values on some survey items, the number of students included in the analysis varies by activity. The numbers of students reported in the 
figure are the number of students who responded to the survey. 

COA = Culinary Operations Academy; DSA = Digital Safari Academy. 



Changes Over Time in Student Outcomes for individuai Students 

Chapter 5 examines the potential of the Exploring Career and College Options (ECCO) pro- 
gram for improving student outcomes, based on whether there has been an improvement over 
time in student outcomes across consecutive cohorts of students at the academies (that is, 
whether students who were enrolled in the grade levels that were exposed to ECCO in spring 
2011 had better outcomes than the previous cohorts of students who were enrolled in these same 
grade levels in spring 2010). Overall, the findings indicate that it is plausible that ECCO may 
have increased students’ participation in college and career exploration activities, as well as 
their career and college awareness, during their first year in the program. 

However, these findings must be interpreted with caution. Improvements over time may 
be attributable to preexisting differences between consecutive cohorts of students at the acade- 
my, rather than to the effect of ECCO. For example, higher college awareness in spring 2011 
than in spring 2010 could be explained by the possibility that the 2011 cohort at an academy 
was more “college aware” at the outset. As discussed in Appendix A, the observed characteris- 
tics of student respondents do differ across survey years. Although the analysis controls for 
these differences, it remains possible that that are still differences in respondents’ unobserved 
characteristics between school years. 

An alternative strategy for evaluating ECCO’s potential for improving student out- 
comes is to look at the change over time for individual students who were enrolled in the 
academies in both survey years (2010 and 2011). “Change” for these students is defined as the 
difference between a survey respondent’s outcomes in 2010 and the same respondent’s out- 
comes in 2011. If ECCO had an effect on student outcomes, then one would expect to see an 
improvement in individual respondents’ outcomes. 

The advantage of this type of analysis — compared with the cohort-level change analy- 
sis in Chapter 5 — is that student-level changes are not confounded with differences in preexist- 
ing characteristics, because the same respondents are being compared in spring 2010 and spring 
2011 (in other words, this analysis implicitly controls for all observed and unobserved time- 
invariant characteristics). That said, the change in student-level outcomes is not without its own 
limitations. Most notably, growth at the individual level could simply be a result of normal 
maturation as students get older and have more relevant experiences. For example, one would 
expect students to become more “college aware” as they get closer to graduation, even if ECCO 
had not been implemented. 

Therefore, the student-level changes in outcomes (presented in this section) and the 
changes at the cohort level (presented in Chapter 5) should be used as complements to each 
other. For stronger evidence that ECCO could have affected student outcomes, changes should 
be observed at both levels. In other words, students who were enrolled in the grade levels that 
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were exposed to ECCO in spring 2011 should have better outcomes than (1) the previous 
cohorts of students enrolled in these grades (cohort-level change) and (2) their own outcomes in 
the previous school year (student-level change). If statistically significant changes are observed 
at both levels, then it is more likely that ECCO had a hand in bringing about these changes. 

The remainder of this section presents estimates of the individual-level changes in stu- 
dent outcomes. For the sake of brevity, the discussion focuses on the student outcomes for 
which there is a statistically significant improvement at the cohort level (Chapter 5) but for 
which the student-level change is not statistically significant. Such a pattern of results reduces 
the likelihood that the cohort-level change over time for this outcome can be plausibly attributa- 
ble to ECCO. In fact, it turns out that there are very few instances of this happening; overall, the 
student-level findings support the conclusions in Chapter 5 and strengthen the plausibility than 
ECCO could have improved student outcomes in respondents’ first year of exposure to the 
program. 

Appendix Table C.3 presents selected characteristics of the students who were used for 
these analyses (that is, students who responded to the survey in both years). For reference, the 
table also shows the characteristics of students who responded to the survey in spring 2010 but 
who did not respond in spring 2011 (that is, the students who are, by definition, excluded from 
this analysis). As seen in this table, students who responded to both surveys are more likely to 
be from a racial/ethnic minority group and were more likely to be enrolled in certain academies 
rather than others. This means that the findings presented in this appendix may not be general- 
izable to all students enrolled in the academies. That said, the findings are nonetheless a useful 
source of information for evaluating ECCO’s potential. 

Participation in Career Exploration Activities 

Appendix Tables C.4 to C.9 present estimates of the student-level changes in participa- 
tion rates in career exploration activities, beginning with a comparison of students with zero and 
two semesters of participation and ending with a comparison of students with two and four 
semesters of participation. Student-level change is evaluated by gauging whether respondents’ 
program participation in spring 2011 was greater than in the previous year (spring 2010). ' 

For students in the first-year academies, for whom zero and two semesters of participa- 
tion in ECCO are compared, the findings support the findings discussed in Chapter 5. That is. 


*The analysis does not look at growth from eleventh to twelfth grade because ECCO was not yet being 
implemented in twelfth grade in spring 2011, so these students’ outcomes could not have been affected by the 
program. 
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The Career Academies Project 
Appendix Table C.3 

Selected Characteristics of Students Who Responded to Both Surveys, 
Compared with Students Who Responded in Spring 2010 Only, 

Pilot Academies 


Characteristic (%) 

Responded 
to Both 
Surveys 

Responded 

Spring 2010 Estimated 
Only Difference 

P- Value 
for Change 

Racial/ethnic minority'* 

91.4 

82.3 

9.1 

0.005 *** 

Male 

47.4 

50.9 

-3.5 

0.465 

Mother's educational attainment 




0.314 

Less than high school 

48.3 

43.8 

4.5 


High school diploma or GED certificate 

30.9 

29.2 

1.7 


Postsecondary credential 

20.2 

27.0 

-6.8 


Ever held back a grade 

23.3 

27.4 

-4.1 

0.332 

Received a D or below in English or math 





in the previous school year 

35.5 

37.9 

-2.3 

0.660 

Academy 




<0.001 *** 

Academy of Art and Technology 

36.8 

40.3 

-3.4 


Center for Hospitality, Tourism, and Marketing 

15.3 

35.4 

-20.1 


Culinary Operations Academy 

8.1 

15.0 

-6.9 


Digital Safari Academy 

39.7 

9.3 

30.4 


Omnibus test (chi-squared = 66.8 ” 




<0.001 *** 

Total number of students'* 

209 

226 




SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on students enrolled in tenth or eleventh grade in spring 2010. Values in the 
"Responded to both surveys" column are the characteristics of students who responded to both the spring 2010 and 
the spring 2011 student survey. Values in the "Responded spring 2010 only" column are the characteristics of 
students who responded to the spring 2010 survey only. The "Estimated difference" column is the difference in 
characteristics between the two groups of students. 

A two-tailed t-test was used to test whether these differences are statistically significant. Statistical significance 
levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

“This includes students who are black, Hispanic, Asian, or "Other" race/ethnicity. 

'’A chi-squared test was used to determine whether there is a systematic difference between the two groups of 
students, across all characteristics included in this table as well as indicators of whether data were missing on each 
of these characteristics. 

“Due to missing values on some survey items and characteristics from the rosters, the number of students 
included in the analysis varies by characteristic. The total numbers of students reported here are for the sample of 
students who responded to the survey. 
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for most of the activities for which there was an increase in participation at the cohort level, 
there was also an improvement at the student level (Appendix Table C.4). 

For students in the second-year academies, for whom two and four semesters of partici- 
pation in ECCO are compared, the findings also generally support those discussed in Chapter 5. 
Appendix Table C.5 shows results of the analysis whereby student-level change is evaluated by 
gauging whether students’ outcomes in spring 2011 (after being exposed to four semesters of 
ECCO) are better than in spring 2010 (after they had experienced two semesters of ECCO). 
These students were enrolled in either tenth or eleventh grade in spring 2010 and had been 
promoted to either eleventh or twelfth grade in spring 2011. 

Estimates of change are presented separately, by grade level (that is, student-level 
changes from tenth to eleventh grade and from eleventh to twelfth grade). When comparing 
these results with those in Chapter 5, the student-level changes from tenth to eleventh grade 
(Appendix Table C.5) should be compared with the cohort-level changes from tenth to eleventh 
grade, while the student-level changes from eleventh to twelfth grade (Appendix Table C.6) 
should be compared with the cohort-level changes for twelfth grade. 

In general, the pattern of results in these tables largely supports the conclusions from 
Chapter 5, that is, that there is not much evidence of an increase in participation rates after the 
first year of implementation. Of the few indicators for which there are cohort-level changes, 
several did not improve at the student-level: 

• Eeam in class about how to write a good resume (tenth and eleventh grade s) 

• Eeam in class about how to act appropriately in a workplace setting (tenth 
and eleventh grades) 

• Do a practice job interview (twelfth grade) 

• Eeam what level of education is needed for various careers (twelfth grade) 

Participation in Coiiege Expioration Activities 

Appendix Tables C.7 to C.9 present similar comparisons of the student-level changes in 
participation in college exploration activities. Again, the results are generally consistent with 
those reported in Chapter 5. Strong growth in participation for most indicators is observed in 
both the cohort-level and the individual-level indicators. But there is little evidence that ECCO 
is associated with additional increases in participation in the second year. 
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The Career Academies Project 
Appendix Table C.4 

Participation in Career Exploration Activities 
During the School Year (2009-2010 to 2010-2011), 
Student-Level Change from Tenth to Eleventh Grade, 
Eirst-Year Academies (AOAT and HTM) 


Indicator (%) 

2009-2010 

2010-2011 

Estimated P- Value 

Change for Change 

Grade level 

Participation in ECCO program 

10 th 
0 semesters 

llth‘’ 
2 semesters 



Attended an activity in which adults came to 
school to talk about jobs or workplace skills 

7.8 

60.8 

52.9 

<0.001 *** 

Did a practice job interview with classmates 
or an adult at school 

17.8 

45.5 

27.7 

<0.001 *** 

Learned in class about how to write a good resume 

42.9 

96.9 

54.1 

<0.001 *** 

Learned in class about how to act appropriately 
in a workplace setting 

65.0 

94.0 

29.0 

<0.001 *** 

Learned in class about how to recognize and respond 
to workplace challenges, such as discrimination 
or harassment 

59.8 

70.6 

10.8 

0.086 * 

Had opportunities to practice or learned 21st- 
century skills 

71.6 

76.5 

4.9 

0.356 

Discussed in class, or with a teacher or counselor, 
what career would be best for you 

23.0 

47.0 

24.0 

<0.001 *** 

Worked with a teacher or counselor to create a plan 
for your future 

18.8 

37.6 

18.8 

<0.001 *** 

Learned about what level of education is needed for 
various careers 

57.4 

64.4 

6.9 

0.210 

Learned about what level of education is needed for 
a career that interests you 

42.6 

57.4 

14.9 

0.006 *** 

Ever visited a work site as part of a school activity to 
learn about what it is like to work there 

6.9 

60.4 

53.5 

<0.001 *** 

Total number of students^ 

109 

109 




(continued) 
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Appendix Table C.4 (continued) 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on students who responded to both the spring 2010 and the spring 201 1 student 
survey. Each spring, students were asked to reflect on their activities during the past school year. Values in the 
"2009-2010" column are the activities of these students during that school year. Values in the "2010-2011" 
column are the activities of these students during that school year. The "Estimated change" column is the 
change in outcomes across school years. Hypothesis testing accounts for repeated observations of students 
across survey years. 

A two-tailed t-test is used for all statistical tests presented in this table. Statistical significance levels are 
indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

"Due to missing values on some survey items, the number of students included in the analysis varies by 
activity. The total numbers of students reported in the table are the number of students who responded to the 
survey in each year. 

’’Seven students (6 percent) were still in tenth grade in spring 201 1 because they had been retained. 

AOAT = Academy of Art and Technology; HTM = Center for Hospitality, Tourism, and Marketing. 
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The Career Academies Project 
Appendix Table C.5 

Participation in Career Exploration Activities 
During the School Year (2009-2010 to 2010-2011), 
Student-Level Change from Tenth to Eleventh Grade, 
Second-Year Academies (COA and DSA) 


Indicator (%) 

2009-2010 

2010-2011 

Estimated P-Value 
Change for Change 

Grade level 

Participation in ECCO program 

10th 
2 semesters 

nth 

4 semesters 



Attended an activity in which adults came to 
school to talk about jobs or workplace skills 

56.3 

81.3 

25.0 

0.104 

Did a practice job interview with classmates 
or an adult at school 

51.0 

40.8 

-10.2 

0.302 

Learned in class about how to write a good resume 

100.0 

95.9 

-4.1 

0.159 

Learned in class about how to act appropriately 
in a workplace setting 

91.8 

98.0 

6.1 

0.175 

Learned in class about how to recognize and respond 
to workplace challenges, such as discrimination 
or harassment 

61.7 

55.3 

-6.4 

0.445 

Had opportunities to practice or learned 21st- 
century skills 

93.9 

91.8 

-2.0 

0.659 

Discussed in class, or with a teacher or counselor, 
what career would be best for you 

40.8 

55.1 

14.3 

0.109 

Worked with a teacher or counselor to create a plan 
for your future 

49.0 

38.8 

-10.2 

0.229 

Learned about what level of education is needed for 
various careers 

llA 

81.3 

4.2 

0.569 

Learned about what level of education is needed for 
a career that interests you 

57.1 

57.1 

0.0 

1.000 

Ever visited a work site as part of a school activity to 
learn about what it is like to work there 

63.3 

83.7 

20.4 

0.017 ** 

Had an internship during this last school year 
or last summer* 

2.1 

10.6 

8.5 

0.097 * 

Total number of students*” 

50 

50 




(continued) 
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Appendix Table C.5 (continued) 

SOURCE; MDRC calculations are from the ECCO spring student surveys. 

NOTES; This analysis is based on students who responded to both the spring 2010 and the spring 2011 student 
survey. Each spring, students were asked to reflect on their activities during the past school year. Values in the 
"2009-2010" column are the activities of these students during that school year. Values in the "2010-2011" 
column are the activities of these students during that school year. The "Estimated change" column is the 
change in outcomes across school years. Hypothesis testing accounts for repeated observations of students 
across survey years. 

A two-tailed t-test is used for all statistical tests presented in this table. Statistical significance levels are 
indicated as follows; *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

‘‘This includes only students in eleventh grade in spring 2010. 

’’Due to missing values on some survey items, the number of students included in the analysis varies by 
activity. The total numbers of students reported in the table are the number of students who responded to the 
survey in each year. 

COA = Culinary Operations Academy; DSA = Digital Safari Academy. 
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The Career Academies Project 
Appendix Table C.6 

Participation in Career Exploration Activities 
During the School Year (2009-2010 to 2010-2011), 
Student-Level Change from Eleventh to Twelfth Grade, 
Second-Year Academies (COA and DSA) 


Indicator (%) 

2009-2010 

2010-2011 

Estimated P-Value 
Change for Change 

Grade level 

Participation in ECCO program 

nth 

2 semesters 

12 th 
4 semesters 



Attended an activity in which adults came to 
school to talk about jobs or workplace skills 

90.9 

72.7 

-18.2 

0.056 * 

Did a practice job interview with classmates 
or an adult at school 

69.4 

63.3 

-6.1 

0.497 

Learned in class about how to write a good resume 

98.0 

86.0 

-12.0 

0.029 ** 

Learned in class about how to act appropriately 
in a workplace setting 

93.9 

89.8 

-4.1 

0.420 

Learned in class about how to recognize and respond 
to workplace challenges, such as discrimination 
or harassment 

77.6 

75.5 

-2.0 

0.799 

Had opportunities to practice or learned 21st- 
century skills 

93.9 

91.8 

-2.0 

0.659 

Discussed in class, or with a teacher or counselor, 
what career would be best for you 

49.0 

53.1 

4.1 

0.687 

Worked with a teacher or counselor to create a plan 
for your future 

53.1 

59.2 

6.1 

0.473 

Learned about what level of education is needed for 
various careers 

79.6 

79.6 

0.0 

1.000 

Learned about what level of education is needed for 
a career that interests you 

71.4 

71.4 

0.0 

1.000 

Ever visited a work site specifically to learn about 
what it is like to work there as part of a school 
activity 

71.4 

42.9 

-28.6 

0.001 *** 

Had an internship during this last school year 
or last summer^ 

4.2 

56.3 

52.1 

<0.001 *** 

Total number of students'’ 

50 

50 




(continued) 
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Appendix Table C.6 (continued) 


SOURCE; MDRC calculations are from the ECCO spring student surveys. 

NOTES; This analysis is based on students who responded to both the spring 2010 and the spring 2011 
student survey. Each spring, students were asked to reflect on their activities during the past school year. 
Values in the "2009-2010" column are the activities of these students during that school year. Values in the 
"2010-2011" column are the activities of these students during that school year. The "Estimated change" 
column is the change in outcomes across school years. Hypothesis testing accounts for repeated observations 
of students across survey years. 

A two-tailed t-test is used for all statistical tests presented in this table. Statistical significance levels are 
indicated as follows; *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

“This includes only students in eleventh grade in spring 2010. 

'’Due to missing values on some survey items, the number of students included in the analysis varies by 
activity. The total numbers of students reported in the table are number of students who responded to the 
survey in each year. 

COA = Culinary Operations Academy; DSA = Digital Safari Academy. 
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The Career Aeademies Projeet 
Appendix Table C.7 

Participation in College Exploration Activities 
Dnring the School Year (2009-2010 to 2010-2011), 
Student-Level Change from Tenth to Elevent Grade, 
First-Year Academies (AO AT and HTM) 


Indicator (%) 

2009-2010 

2010-2011 

Estimated P- Value 

Change for Change 

Grade level 

Participation in ECCO program 

JOth Util’’ 

0 semesters 2 semesters 



Talked with a teacher or other adult at school about college 

44.4 

68.9 

24.4 

<0.001 *** 

Talked with a parent or other family member about college 

75.6 

75.6 

0.0 

1.000 

Attended a college fair 

7.8 

27.8 

20.0 

<0.001 *** 

Attended a presentation where college representatives 
or graduates talked about college 

7.8 

27.8 

20.0 

<0.001 *** 

Looked up information about colleges on the Internet 

48.9 

81.1 

32.2 

<0.001 *** 

Visited a 4-year college campus 

28.9 

78.9 

50.0 

<0.001 *** 

Visited a community college or career-technical school 

32.2 

45.6 

13.3 

0.033 ** 

Talked with parents, teacher, or counselor about 
how to pay for college 

27.8 

51.1 

23.3 

<0.001 *** 


Took the P-SAT (or signed up to take it this summer) 

Took the SAT or ACT (or signed up to take it this summer) 

Applied to a community college or career-technical program 

Applied to a 4-year college 

Did anything else to prepare for college 

Total number of students'* 109 109 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on students who responded to both the spring 2010 and the spring 201 1 student 
survey. Each spring, students were asked to reflect on their activities during the past school year. Values in the 
"2009-2010" column are the activities of these students during that school year. Values in the "2010-2011" column 
are the activities of these students during that school year. The "Estimated change" column is the change in 
outcomes across school years. Hypothesis testing accounts for repeated observations of students across survey years. 

A two-tailed t-test is used for all statistical tests presented in this table. Statistical significance levels are indicated 
as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

“Due to missing values on some survey items, the number of students included in the analysis varies by activity. 
The total numbers of students reported in the table are the number of students who responded to the survey in each 
year. 

'“Seven students (6 percent) were still in tenth grade in spring 2011 because they had been retained. 

AO AT = Academy of Art and Technology; HTM = Center for Hospiality, Tourism, and Marketing. 
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The Career Academies Project 
Appendix Table C.8 

Partieipation in College Exploration Aetivities 
During the Sehool Year (2009-2010 to 2010-2011), 
Student-Level Change from Tenth to Eleventh Grade, 
Seeond-Year Aeademies (COA and DSA) 


Indicator (%) 

2009-2010 

2010-2011 

Estimated P- Value 
Change for Change 

Grade level 

Participation in ECCO program 

10th nth’’ 

2 semesters 4 semesters 



Talked with a teacher or other adult at school about college 

70.5 

81.8 

11.4 

0.200 

Talked with a parent or other family member about college 

75.0 

113 

2.3 

0.743 

Attended a college fair 

16.7 

28.6 

11.9 

0.096 * 

Attended a presentation where eollege representatives 
or graduates talked about college 

16.7 

28.6 

11.9 

0.096 * 

Looked up information about colleges on the Internet 

69.8 

95.3 

25.6 

0.001 *** 

Visited a 4-year college campus 

83.3 

100.0 

16.7 

0.007 *** 

Visited a eommunity college or career-technical school 

61.9 

73.8 

11.9 

0.168 

Talked with parents, teacher, or counselor about 
how to pay for college 

38.1 

52.4 

14.3 

0.110 


Took the P-SAT (or signed up to take it this summer) 

Took the SAT or ACT (or signed up to take it this summer) 

Applied to a community eollege or eareer-technical program 

Applied to a 4-year college 

Did anything else to prepare for eollege 

Total number of students^ 50 50 


SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: This analysis is based on students who responded to both the spring 2010 and the spring 2011 student 
survey. Each spring, students were asked to reflect on their activities during the past school year. Values in the 
"2009-2010" eolumn are the activities of these students during that sehool year. Values in the "2010-2011" column 
are the activities of these students during that school year. The "Estimated change" column is the ehange in 
outcomes across school years. Hypothesis testing aecounts for repeated observations of students across survey years. 

A two-tailed t-test is used for all statistical tests presented in this table. Statistical significance levels are indicated 
as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may eause slight discrepaneies in ealculating sums and differences. 

““Due to missing values on some survey items, the number of students included in the analysis varies by activity. 
The total numbers of students reported in the table are the number of students who responded to the survey in each 
year. 

'’Seven students (6 pereent) were still in tenth grade in spring 2011 because they had been retained. 

COA = Culinary Operations Academy; DSA = Digital Safari Aeademy. 
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The Career Academies Project 
Appendix Table C.9 

Participation in College Exploration Activities 
During the School Year (2009-2010 to 2010-2011), 
Student-Level Change from Eleventh to Twelfth Grade, 
Second-Year Academies (COA and DSA) 


Indicator (%) 

2009-2010 

2010-2011 

Estimated P-Value 
Change for Change 

Grade level 

Participation in the ECCO program 

11th 12 th 

2 semesters 4 semesters 



Talked with a teacher or other adult at school about college 

80.9 

91.5 

10.6 

0.133 

Talked with a parent or other family member about college 

83.0 

87.2 

4.3 

0.485 

Attended a college fair 

44.7 

61.7 

17.0 

0.044 ** 

Attended a presentation where college representatives 
or graduates talked about college 

44.7 

61.7 

17.0 

0.044 ** 

Looked up information about colleges on the Internet 

95.7 

93.6 

-2.1 

0.569 

Visited a 4-year college campus 

95.7 

91.5 

-4.3 

0.406 

Visited a community college or career-technical school 

57.4 

83.0 

25.5 

<0.001 *** 

Talked with parents, teacher, or counselor about 
how to pay for college 

66.0 

87.2 

21.3 

0.011 ** 

Took the P-SAT (or signed up to take it this summer)'* 

43.5 

60.9 

17.4 

0.044 ** 

Took the SAT or ACT (or signed up to take it this summer)" 


75.5 


<0.001 *** 

Applied to a community college or career-technical program" 


57.1 


<0.001 *** 

Applied to a 4-year college" 


75.5 


<0.001 *** 

Did anything else to prepare for college" 


75.5 


<0.001 *** 

Total number of students'* 

50 

50 




SOURCE: MDRC calculations are from the ECCO spring student surveys. 


NOTES: This analysis is based on students who responded to both the spring 2010 and the spring 2011 student 
survey. Each spring, students were asked to reflect on their activities during the past school year. Values in the 
"2009-2010" column are the activities of these students during that school year. Values in the "2010-2011" column 
are the activities of these students during that school year. The "Estimated change" column is the change in 
outcomes across school years. Hypothesis testing accounts for repeated observations of students across survey years. 

A two-tailed t-test is used for all statistical tests presented in this table. Statistical significance levels are indicated 
as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

““This includes only studnets in eleventh grade in spring 2010. 

'’Due to missing values on some survey items, the number of students included in the analysis varies by activity. 
The total numbers of students reported in the table are the number of students who responded to the survey in each 
year. 

COA = Culinary Operations Academy; DSA = Digital Safari Academy. 
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Outcome Measures 

Appendix Tables C.IO to C.13 present the individual-level comparisons for the short- 
term outcomes discussed in Chapter 5. In general the pattern of results again largely supports 
the conclusions from Chapter 5, that is, that there is not much evidence of improvement in 
student outcomes after the first two semesters. As Appendix Table C.IO shows, individual-level 
and cohort-level growth are seen in two of the four indicators for the first-year academies but 
are not seen for the second-year academies. For some outcomes, improvement at the cohort 
level was not observed at the student level or vice versa. 


The Career Academies Project 
Appendix Table C.IO 

Student-Level Change in Short-Term Outcomes, 
First-Year Academies (AOAT and HTM) 


Outcome 

Spring 

2010 

Spring 

2011 

Estimated 

Change 

Effect 

Size 

P -Value 
for Change 

Change from 10th to 11th grade 






Grade level 

10 th 

llth’’ 




Participation in ECCO program 

0 semesters 

2 semesters 




Engagement 

2.92 

2.94 

0.02 

0.03 

0.700 

Career awareness 

2.77 

3.03 

0.26 

0.33 

0.001 *** 

College awareness 

2.63 

2.89 

0.26 

0.33 

0.002 *** 

Self-assessed 2 1 st-century skills 

3.25 

3.40 

0.15 

0.29 

0.003 *** 

Total number of students'* 

109 

109 





SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: The student outcomes presented in this table are calculated by averaging across a student's responses to 
relevant survey items. These survey items (and the resulting outcomes) are on a 4-point scale (1 = not at all true; 2 
= not very true; 3 = sort of true; 4 = very true). This analysis is based on students who responded to both the spring 
2010 and the spring 2011 student survey. The "Spring 2010" column presents the outcomes of these students in 
spring 2010. The "Spring 2011" column presents the outcomes of these students in spring 2011. The "Estimated 
change" column is the change in outcomes for these students from spring 2010 to spring 2011. Elypothesis testing 
accounts for repeated observations of students across survey years. 

A two-tailed t-test is used for all statistical tests presented in this table. Effect sizes are based on the standard 
deviation of an outcome for all survey respondents in spring 2010 and spring 2011. Statistical significance levels 
are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

"Due to missing values on some survey items, the number of students included in the analysis varies by 
outcome. The total numbers of students reported in the table are the number of students who responded to the 
survey. 

'’Seven students (6 percent) were still in tenth grade in spring 2011 because they had been retained. 

AOAT = Academy of Art and Technology; HTM = Center for Hospitality, Tourism, and Marketing. 
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The Career Academies Project 
Appendix Table C.ll 

Student-Level Change in Short-Term Outcomes, 
Second-Year Academies (COA and DSA) 



Spring 

Spring 

Estimated 

Effect 

P -Value 

Outcome 

2010 

2011 

Change 

Size for Change 

Change from 10th to 11th grade 






Grade level 

10th 

nth 




Participation in ECCO program 

2 semesters 

4 semesters 




Engagement 

2.84 

2.80 

-0.04 

-0.07 

0.580 

Career awareness 

2.95 

2.84 

-0.12 

-0.15 

0.306 

College awareness 

2.90 

2.77 

-0.13 

-0.16 

0.283 

Self-assessed 21st-century skills 

3.20 

3.30 

0.10 

0.21 

0.115 

Total number of students^ 

50 

50 




Change from 11th to 12th grade 






Grade level 

nth 

12th 




Participation in ECCO program 

2 semesters 

4 semesters 




Engagement 

3.18 

3.09 

-0.10 

-0.18 

0.181 

Career awareness 

3.07 

3.14 

0.07 

0.09 

0.444 

College awareness 

3.00 

3.48 

0.48 

0.61 

<0.001 *** 

Self-assessed 21st-century skills 

3.45 

3.39 

-0.06 

-0.12 

0.314 

Total number of students^ 

50 

50 





SOURCE; MDRC calculations are from the ECCO spring student surveys. 

NOTES: The student outcomes presented in this table are calculated by averaging across a student's responses 
to relevant survey items. These survey items (and the resulting outcomes) are on a 4-point scale (1 = not at all 
true; 2 = not very true; 3 = sort of true; 4 = very true). This analysis is based on students who responded to both 
the spring 2010 and the spring 2011 student survey. The "Spring 2010" column presents the outcomes of these 
students in spring 2010. The "Spring 2011" column presents the outcomes of these students in spring 2011. The 
"Estimated change" column is the change in outcomes for these students from spring 2010 to spring 
2011. Hypothesis testing accounts for repeated observations of students across survey years. 

A two-tailed t-test is used for all statistical tests presented in this table. Effect sizes are based on the standard 
deviation of an outcome for all survey respondents in spring 2010 and spring 2011. Statistical significance 
levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

'“Due to missing values on some survey items, the number of students included in the analysis varies by 
outcome. The total numbers of students reported in the table are the number of students who responded to the 
survey. 

COA = Culinary Operations Academy; DSA = Digital Safari Academy. 
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The Career Academies Project 
Appendix Table C,12 


Student-Level Change in Beliefs About Connection Between School and Future, 
First- Year Academies (AOAT and HTM) 


Outcome 

Spring 

2010 

Spring 

2011 

Estimated 

Change 

P-Value 
for Change 

Chanse from 10th to 11th srade 

Grade level 

Participation in ECCO program 

10 th 
0 semesters 

11th'’ 
2 semesters 



There is a connection between school and 
what I will do when I finish school 

46.7 

56.1 

9.3 

0.105 

What is learned in school is important for later 
in life 

78.5 

80.4 

1.9 

0.685 

Total number of students'* 

109 

109 




SOURCE: MDRC calculations are from the ECCO spring student surveys. 

NOTES: The student outcomes presented in this table are calculated by averaging across a student's 
responses to relevant survey items. These survey items (and the resulting outcomes) are on a 4-point scale 
(1 = not at all true; 2 = not very true; 3 = sort of true; 4 = very true). This analysis is based on students 
who responded to both the spring 2010 and the spring 2011 student survey. The "Spring 2010" column 
presents the outcomes of these students in spring 2010. The "Spring 2011" column presents the outcomes 
of these students in spring 2011. The "Estimated change" column is the change in outcomes for these 
students from spring 2010 to spring 2011. Hypothesis testing accounts for repeated observations of 
students across survey years. 

A two-tailed t-test is used for all statistical tests presented in this table. Effect sizes are based on the 
standard deviation of an outcome for all survey respondents in spring 2010 and spring 2011. Statistical 
significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

““Due to missing values on some survey items, the number of students included in the analysis varies by 
outcome. The total numbers of students reported in the table are the number of students who responded to 
the survey. 

'’Seven students (6 percent) were still in tenth grade in spring 2011 because they had been retained. 

AOAT = Academy of Art and Technology; HTM = Center for Hospitality, Tourism, and Marketing. 
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The Career Aeademies Projeet 
Appendix Table C.13 


Student-Level Change in Beliefs About Connections Between School and Future, 
Second-Year Academies (COA and DSA) 


Outcome (%) 

Spring 

2010 

Spring 

2011 

Estimated P-Value 

Change for Change 

Chanse from 10th to 11th grade 





Grade level 

Participation in ECCO program 

10 th 
2 semester 

11th 
4 semesters 



There is a connection between school and 
what I will do when I finish school 

50.0 

58.3 

8.3 

0.351 

What is learned in school is important for later 
in life 

83.3 

72.9 

-10.4 

0.133 

Total number of students^ 

50 

50 



Change from 11th to 12th grade 





Grade level 

Participation in ECCO program 

11th 
2 semester 

12th 
4 semesters 



There is a connection between school and 
what I will do when I finish school 

68.0 

74.0 

6.0 

0.322 

What is learned in school is important for later 
in life 

85.4 

91.7 

6.3 

0.182 

Total number of students^ 

50 

50 




SOURCE; MDRC calculations are from the ECCO spring student surveys. 

NOTES; The student outcomes presented in this table are calculated by averaging across a student's responses 
to relevant survey items. These survey items (and the resulting outcomes) are on a 4-point scale (1 = not at all 
true; 2 = not very true; 3 = sort of true; 4 = very true). This analysis is based on students who responded to 
both the spring 2010 and the spring 2011 student survey. The "Spring 2010" column presents the outcomes of 
these students in spring 2010. The "Spring 2011" column presents the outcomes of these students in spring 
2011. The "Estimated change" column is the change in outcomes for these students from spring 2010 to spring 
2011. Hypothesis testing accounts for repeated observations of students across survey years. 

A two-tailed t-test is used for all statistical tests presented in this table. Effect sizes are based on the 
standard deviation of an outcome for all survey respondents in spring 2010 and spring 2011. Statistical 
significance levels are indicated as follows; *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

^Due to missing values on some survey items, the number of students included in the analysis varies by 
outcome. The total numbers of students reported in the table are the number of students who responded to the 
survey. 

COA = Culinary Operations Academy; DSA = Digital Safari Academy. 
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Statistical Models 


This section presents the statistical models used to estimate the cohort-level change in Chapter 
5, the student-level changes presented in this appendix, and the student-level changes for interns 
compared with nonintems (Chapter 5). 


Cohort-Level Changes Over Time 


The cohort-level changes over time (Chapter 5) are estimated by fitting the following 
multilevel model to the sample of survey respondents:^ 


Yi = pYRZOlli + (jOggSITEgiGRADEgi + Si 


Where 


Yi = Participation or outcome for student i 

YR2011i = Indicator equal to 1 for students surveyed in spring 2011 and 

equal to 0 for students surveyed in spring 2010 

Xjc = A set of K student characteristics for student 

SITEs = A set of S dummy indicators for the academy in which student i is 

enrolled 

GRADEg = A set of G dummy indicators for the grade level in which student i 
is enrolled"^ 

£i = Random variation in outcome Y for student i 

In this model, ^ is the estimated cohort-level change over time (from spring 2010 to 
spring 2011). The academy indicators (SITE) account for the fact that the data are clustered 
(students are nested within schools), which makes it possible to obtain the right standard errors 


^Individual-level change is simply the difference in outcomes from 2010 to 201 1. There is no need to ad- 
just for differences in students’ characteristics because this type of before-and -after analysis implicitly controls 
for all observed and unobserved characteristics that are fixed over time. 

^The following characteristics are included in the model: whether the student is in a racial/ethnic minority 
group, the student’s gender, the educational attainment of the student’s mother, whether the student was ever 
held back, whether the student received a D or below in English or math in the previous school year, and 
missing indicators for each of these characteristics. 

"^These dummy indicators for grade level are included only for analyses that pool students in different 
grade levels (that is, analyses that pool tenth- and eleventh-grade students). 
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and p-values for hypothesis testing. The analysis is conducted separately for each grade level 
(tenth or eleventh grade and twelfth grade) and for each group of academies (first-year acade- 
mies and second-year academies). 

Student-Level Changes Over Time 

Student-level changes over time are estimated by fitting the following multilevel model 
to the sample of students who responded to both surveys: 


Yit = a + pYRIOllit +Si+ Tit 


Where 


Yit = Participation or outcome for student i in time t (spring 20 1 0 or 

spring 2011) 

YR2011it = Indicator equal to 1 if a student’s outcome is measured in spring 
2011 and equal to 0 if it is measured in spring 2010 

£j = Random variation in outcomes Y across students (between-student 

error term) 

Tit ^ Random variation in outcome Y across time for student i (within-student 
error term) 

In this model, (i is the estimated student-level change over time (from spring 2010 to 
spring 2011) for students in the sample. The multilevel sttucture of the error term accounts for 
the fact that the data are longitudinal and, therefore, are clustered (time points are nested within 
students), which makes it possible to obtain the right standard errors and p-values for hypothesis 
testing. 

The student-level changes are estimated separately for students in the first-year acade- 
mies and for those in the second-year academies. The findings for the first-year academies 
represent student-level growth during the first year of students’ exposure to ECCO, while 
findings for second-year academies provide information on student-level change during 
students’ second year of exposure to the program. 
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